SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. Acknowledgements
  7. REFERENCES

Objective

The American College of Rheumatology (ACR) 1987 criteria for rheumatoid arthritis (RA) can be applied in 2 formats, a standard “x/y” list and a decision tree. This study evaluated the performance of the decision tree compared with the list approach in the ascertainment of RA in subjects with new-onset inflammatory polyarthritis (IP) over the first 5 years of observation. Moreover, the use of clinical surrogates to substitute for missing rheumatoid factor (RF) and radiologic erosion data was assessed for validity and for its influence on the resulting RA prevalence estimates.

Methods

In this population-based prospective study, 848 subjects with new-onset IP were interviewed and examined at baseline, with followup at 1, 2, 3, and 5 years. RF and erosion status were determined at prespecified time points. The list criteria were applied cumulatively, while the decision tree was applied cross-sectionally using either data surrogates or the actual reported data. RA prevalence in the 848 subjects and agreement in classification between the 2 methods was assessed at each time point. The influence of using clinical surrogates on RA prevalence estimates at 5 years and the agreement between surrogate and real results were also analyzed.

Results

At baseline, RA prevalence was higher using the decision tree compared with the list approach (63% versus 47%; P = 0.0001); by 5 years of followup, RA estimates were approximately equal (69% versus 72%) and agreement between the approaches was good (κ = 0.67). The use of surrogates had little influence on RA prevalence at 5 years, although substitution of metacarpophalangeal joint swelling for erosion produced a higher RA prevalence estimate (78% versus 70%). Although there was only weak agreement between surrogate and real data, the use of the surrogate data provided good to very good agreement between the approaches in categorizing subjects as RA positive (κ = 0.61–0.72).

Conclusion

Over 5 years, the 2 formats of the ACR criteria for RA performed similarly, with no important differences between them. The use of surrogates for missing radiologic and serologic data did not have any major influence on disease classification. Although the RA criteria were not originally derived from subjects with early disease in a population setting, this study shows that the use of the decision tree approach with the option of substituting clinical surrogates for missing laboratory data is an appropriate alternative to the conventional list approach.

The American College of Rheumatology (ACR; formerly, the American Rheumatism Association) 1987 criteria for the classification of rheumatoid arthritis (RA) were initially devised to distinguish subjects with established RA from subjects with other established rheumatic conditions (1) for the purposes of clinical and epidemiologic studies. Based on the prevalence of key disease features in groups of subjects with and without RA, 2 sets of criteria were proposed. One approach requires the presence of at least 4 of a list of 7 symptoms (Table 1). The other approach is based on a decision tree, in which classification of subjects as RA positive requires membership in 1 of 5 subgroups based on different combinations of the disease features studied (Figure 1).

Table 1. Standard list format of the American College of Rheumatology 1987 criteria for the classification of rheumatoid arthritis*
CriterionDefinition
  • *

    PIP = proximal interphalangeal; MCP = metacarpophalangeal; MTP = metatarsophalangeal.

Morning stiffnessMorning stiffness in and around the joints, lasting at least 1 hour
Arthritis of ≥3 joint areasSoft-tissue swelling or fluid in at least 3 of the following areas: the left or right PIP, MCP, wrist, elbow, knee, ankle, or MTP joints
Arthritis of the hand jointsSwelling of wrist, MCP, or PIP joints
Symmetric arthritisSimultaneous involvement of the same joint areas (as above) on both sides of the body (at least 50% of affected joint areas affected symmetrically)
Rheumatoid nodulesSubcutaneous nodules present
Rheumatoid factorDetected by a method that yields positive findings in <5% of normal controls
Radiographic changesErosions or unequivocal bony decalcification localized to the joints of the hands and wrists
thumbnail image

Figure 1. Decision tree format of the American College of Rheumatology 1987 criteria for the classification of rheumatoid arthritis (RA), as applied to patients with inflammatory polyarthritis (IP). MCP = metacarpophalangeal; RF = rheumatoid factor.

Download figure to PowerPoint

Using the decision tree, if the status of erosions and/or rheumatoid factor (RF) is unknown, the surrogate variables of swelling of the metacarpophalangeal (MCP) joints (for radiologic erosions) and swelling of the wrist (for RF positivity) could be substituted without loss of validity in the original data set (1). This substitution option is potentially helpful in epidemiologic and clinical studies that rely on retrospective chart review, since the results of these investigations are frequently unavailable. There have, however, been few published reports comparing the performance of the list approach with that of the tree approach, nor have there been many studies assessing the influence, if any, of making these substitutions. Studies have suggested that the decision tree strategy is more sensitive (2). However, it is likely that any disagreement between the tree and list definitions is due to the use of surrogates, since the surrogate option is not applicable in the list definition. With the list approach, missing data on RF status or radiologic findings have to be considered as negative, reducing the RA prevalence estimates and the number of subjects available for study.

An additional concern is whether the ACR 1987 criteria, in either format, are appropriate for classifying subjects shortly after disease onset. The criteria, when originally developed, were applied to subjects with longstanding disease and are considered suitable for selecting such subjects for inclusion in, for example, clinical trials. The ability of the criteria to identify individuals with new-onset inflammatory polyarthritis (IP) who could be classifiable as having RA is more problematic (3). Thus, it has been shown that applying the ACR criteria to patients with new-onset IP is not useful for predicting persistent, disabling, or erosive disease (4). One reason for this may be that a number of the individual criteria, for example, nodules and erosions that may not be present initially, develop over time. Therefore, the performance of the criteria changes with disease duration (5). In contrast, other disease features, such as joint swelling, vary with time (and in response to therapy). It is therefore advisable to use a cumulative approach to disease classification, and as a consequence, in any prospective study of new-onset IP, the proportion of subjects classified as having RA will also increase over time.

In this study, we compared the use of the decision tree with the use of the list approach for the classification of subjects as RA positive among individuals with new-onset IP who were followed up for 5 years. We compared, over the period of followup, the proportion of subjects classified as having RA by each definition, as well as the agreement between the 2 approaches. In addition, we investigated the effect of using the surrogate variables on the tree definition by addressing 2 questions: 1) Compared with the use of actual laboratory and radiologic data, what was the influence of using surrogate variables on RA prevalence? 2) Were the same subjects identified as having RA when using the surrogate variables?

PATIENTS AND METHODS

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. Acknowledgements
  7. REFERENCES

Design

A population-based prospective study was used to recruit patients with new-onset IP. Subjects were followed up annually, and the ACR 1987 classification criteria for RA (1) were applied at each visit.

Study population

The study sample was composed of patients referred to the UK Norfolk Arthritis Register (NOAR). This register provides information on approximately one-half million adult residents within the former Norwich Health Authority, as described in detail elsewhere (6). Briefly, all new patients who were under the care of a primary care physician, who had resided within the NOAR area since 1990, and who had IP (defined as swelling of at least 2 joints persisting for a minimum of 4 weeks) were eligible for recruitment. Patients are followed up annually. The present analysis uses data from all 848 subjects recruited between 1990 and 1995 who were followed up for 5 years.

Baseline assessment

Following recruitment, patients were interviewed and examined by one of a team of specially trained research nurses. The presence of the following features included in the ACR 1987 criteria for RA were ascertained: 1) ever having morning stiffness of at least 60 minutes' duration; 2) soft tissue swelling in any of the following joint areas: proximal interphalangeal (PIP) joints, MCP joints, wrists, elbows, knees, ankles, and metatarsophalangeal joints; 3) at least 50% of the joint areas (as listed above) being involved symmetrically; and 4) rheumatoid nodules. A serum sample was assayed for RF by tube latex dilution technique, with a titer of ≥1:40 considered to be positive. Radiographs were not obtained at baseline because the subjects were considered unlikely to have erosions at this time point.

Determination of RA status at baseline

Subjects were classified at baseline as having RA according to the list definition if they satisfied at least 4 criteria from the list. Given the absence of radiographs, all subjects were assumed to be erosion-free for the purposes of applying the criteria in the list format. In contrast, in keeping with the published criteria rules for defining RA using the decision tree format (1), the presence or absence of MCP joint swelling was used as a surrogate for missing radiologic data for all subjects, and wrist swelling was used when an RF measurement was not available.

Followup assessments

Subjects were reassessed by the research nurse after 1, 2, 3, and 5 years. A blood sample for RF measurement was obtained at year 1, at year 2 if the previous result was negative, and at year 5 in all subjects. At year 1, patients were selected to undergo radiography of the hands and feet if the detection of erosions would permit classification as RA positive based on the list approach, i.e., for those subjects who had at least 3 of the individual criteria. Further radiographs were obtained at year 2 if the previous films did not show erosions. All subjects were asked to undergo radiographic assessments at 5 years. All radiographs were scored, by 2 observers independently, for the presence of erosions using the Larsen grading system (7). The presence of any erosion (i.e., a Larson score of ≥2 for any joint) constituted a positive finding. In the case of disagreement, a third observer arbitrated the decision.

Determination of RA status during followup

Subjects were considered positive for any of the 7 individual criteria features in the list format definition of RA if they had ever had a positive result for that criterion at any visit. Thus, the prevalence of RA using this definition could only increase with time. Similarly, the highest result at any time for each of the criteria in the tree definition was used to define RA status at each visit. However, subjects who had a positive result for a surrogate variable and who were subsequently found to be negative for the original variable could change from being RA positive to being RA negative, as would appear to be suggested from the published rules.

Statistical analysis

The cumulative prevalence of RA within the cohort (i.e., the proportion of the whole IP cohort who satisfied the criteria for RA) was calculated at each time point, using both the list and the tree approaches. McNemar's test was used to determine whether differences in RA prevalence were statistically significant. The agreement between the 2 classification systems was assessed separately at each time point, using a kappa statistic. The validity of the surrogates in the tree approach was assessed at 5 years, at which point all subjects had current radiographs and at least 1 RF result. Kappa statistics were used to assess the agreement between the prevalence estimates of the real results from these tests and those derived from the permitted surrogates. Finally, the impact of using the surrogates compared with the actual values was assessed by ascertaining the agreement in subjects' classification as having RA. All analyses were performed using Stata software, version 8.2 (Chicago, IL).

RESULTS

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. Acknowledgements
  7. REFERENCES

The frequencies, at baseline, of the individual features used to classify subjects as RA positive by either approach are shown in Table 2. In this primary care–derived cohort of subjects with IP, the prevalence of RF positivity at baseline was low (25%) compared with what would be expected in a hospital cohort of RA subjects, whereas the prevalence of its surrogate, wrist swelling, was higher (40%). The prevalence of erosions, based on its surrogate of MCP swelling, was substantial at baseline, at 66%, which is far in excess of the expected prevalence of radiologic erosions at baseline.

Table 2. Prevalence of individual rheumatoid arthritis criteria features at baseline*
FeatureValue
  • *

    Except where indicated otherwise, values are the number (%) of patients (n = 848). MCP = metacarpophalangeal.

  • Surrogate for rheumatoid factor positivity.

  • Surrogate for erosions.

Male275 (32)
Age at onset, mean ± SD years53 ± 15
Morning stiffness568 (67)
Arthritis of ≥3 joint areas561 (66)
Arthritis of the hand joints686 (81)
Symmetric arthritis537 (63)
Rheumatoid nodules55 (6)
Rheumatoid factor positive209 (25)
Swelling of the wrist338 (40)
Radiographic erosionNot tested
Swelling of the MCP joints559 (66)

The cumulative prevalence of RA at each assessment is given in Table 3. The prevalence was initially higher using the tree definition; however, this was not surprising, because the tree approach permits a prevalent clinical surrogate as a replacement for a presumed low prevalence of erosions. The difference between the 2 approaches decreased substantially over time, with no important differences at any of the subsequent time points. There was also some evidence that agreement in the classification of RA by the 2 approaches also increased over time. The agreement at baseline was already reasonable (a kappa statistic of ≥0.6 is conventionally considered very good [8]), but there was a continuing modest increase in agreement with each year. As mentioned above, one problem with applying the tree definition (as originally described) cumulatively is that subjects may change from being RA positive to RA negative if the surrogate variable is positive and the original variable, when measured later, is negative. This accounts for the slight fall in the prevalence of RA between baseline and 1 year, using the tree definition.

Table 3. Influence of disease duration on the prevalence of rheumatoid arthritis (RA), as determined using the list and decision tree approaches
YearRA prevalence, %PExpected agreement, %*Observed agreement, %Kappa
ListTreeBoth
  • *

    Expected agreement refers to the proportion of subjects expected based on the marginal totals in both groups, assuming only a random relationship.

  • Observed agreement refers to the proportion of subjects for which there was agreement by both methods on either the presence or the absence of RA.

Baseline4763440.000149790.59
15961510.1352810.60
26565560.955820.61
36767590.656840.63
57269640.0958860.67

The effect of using surrogate variables was investigated in the 636 subjects who had both RF positivity and radiologic damage at 5 years. As anticipated from the baseline data, even at 5 years, the prevalence estimates of the clinical surrogates for RF positivity and erosions were both much higher than the prevalences based on the real results (Table 4). Furthermore, the agreement between the use of the real data and the use of the surrogate data was poor for both features. However, when used in the tree algorithm, the use of the surrogate variables yielded prevalences of RA that were similar to those derived from the real data (Table 5). Using MCP joint swelling as a surrogate for erosions led to a small increase in RA prevalence (78% versus 70%). Furthermore, in comparing the use of surrogates with the use of the original variables, the agreement between classifications of subjects as RA positive was very good when only 1 surrogate was used, but substantially worse when both surrogates were used (Table 5).

Table 4. Prevalence of the original (real) variables compared with the surrogate variables at 5 years among 636 subjects*
Original variableSurrogate variablePrevalence of real variable, %Prevalence of surrogate variable, %PKappa
  • *

    MCP = metacarpophalangeal; RF = rheumatoid factor.

  • Agreement in prevalence estimates.

ErosionsMCP joint swelling46780.00010.25
RF positivityWrist swelling40580.00010.13
Table 5. Effect of using surrogate variables on the tree definition of rheumatoid arthritis (RA) at 5 years*
Surrogates usedPrevalence of RA, %Expected agreement, %Observed agreement, %Kappa
  • *

    See Table 4 for other definitions.

  • Agreement in classification of subjects as RA positive.

None70
Wrist swelling for RF6757900.77
MCP joint swelling for erosions7861910.77
Both surrogates7761840.59

The final test of the performance of the surrogates was to assess the agreement in the classification of RA at 5 years using the list and the tree approaches, but with calculation of the latter using no surrogates, either surrogate, or both surrogates. The agreement between the list approach and these different variations on the tree approach is shown in Table 6. The use of wrist swelling as a surrogate for RF positivity leads to a weakening of agreement, although interestingly, the agreement between approaches was very good when both surrogates were used or when just MCP joint swelling was used as a substitute for erosions.

Table 6. Effect of using surrogate variables on agreement between the tree definition and list definition of rheumatoid arthritis (RA) at 5 years*
Surrogates usedExpected agreement, %Observed agreement, %Kappa
  • *

    See Table 4 for other definitions.

  • Agreement in classification of subjects as RA positive.

None59870.69
Wrist swelling for RF58840.61
MCP joint swelling for erosions63890.72
Both surrogates63880.69

DISCUSSION

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. Acknowledgements
  7. REFERENCES

The ACR 1987 criteria for RA have been widely used and accepted since their publication, but the performance of the decision tree approach has never been fully investigated. The advantage of the tree approach conceptually is that it gives different weights to the different constituent variables, reflecting their relative discriminatory performance. The other advantage is that, by allowing surrogate variables for missing laboratory data, they reflect the real-life situation, particularly in studies that rely on retrospective chart review. The tree approach, however, has not been as popular in use, given the greater ease in applying the 4 of 7 list rule. The application of criteria is probably most useful clinically in early disease, and thus, a close examination of the relative merits of the 2 formats is best carried out in an inception cohort of patients with IP, as has been undertaken in the present study.

The first key finding was that the prevalence of RA using the tree approach was substantially higher at presentation, probably because of the option to use surrogates. However, in the subsequent 5 years, the RA prevalence estimates using the 2 approaches converged. Nevertheless, the individuals classified as having RA by the 2 methods were not the same, although the agreement was, according to a standard interpretation (8), good. A key question is, which of the 2 methods produces a closer approximation to “real” RA? The gold standard of physician opinion is probably too influenced by knowledge of the criteria to be of value.

An alternative approach is to consider how subjects classified by one or both of the approaches vary in their prognosis (construct validity). For this purpose, the use of erosions is limited because of the relative contribution of erosions to disease classification. As an indicator of construct validity, we therefore analyzed the mean Health Assessment Questionnaire (HAQ) scores (9) at 5 years in those subjects who were RA positive only by the list, only by the decision tree, and by both the list and the decision tree at presentation. The results for the 3 groups were mean HAQ scores of 1.3, 0.9, and 1.3, respectively, suggesting a lower specificity for severe disease among those subjects who were classified by the tree approach only. Reassuringly, those subjects who, at baseline, were RA negative by both the tree and the list approaches had a very low 5-year mean HAQ score of 0.31. Thus, the overall conclusion is that the 2 methods generate similar RA prevalence estimates, although use of the list may identify, albeit modestly, a more severely affected group. The erosions data, although less reliable for the reasons given earlier, produced a similar picture, with RA prevalences (determined using the presence of erosions at 5 years) of 25%, 56%, 45%, and 55% in subjects who were RA positive at baseline by neither definition, the list definition only, the tree definition only, and both definitions, respectively.

The second key finding was that the use of surrogates for erosions and RF positivity, albeit agreeing weakly with the real results, did not have a major impact on the overall RA prevalence estimates. Indeed, the substitution of wrist swelling for RF positivity actually produced a modest decrease in RA prevalence, whereas the substitution of MCP joint swelling for erosions had the opposite and numerically greater effect. The interesting finding was that the agreement between the list and tree approaches was the same regardless of whether one or both surrogates were used. It is reassuring that the original surrogates performed well in this completely different cohort.

The use of surrogates may be most useful at presentation, since clinical studies requiring contemporary recruitment of subjects later during the disease course clearly do not require the use of surrogates because the real results can be readily and easily obtained. In contrast, the observations from these analyses are of value to those undertaking studies that rely on retrospective chart review, in which laboratory and radiographic data are frequently missing. Indeed, this type of experience was the rationale for their inclusion in the original ACR study (1).

One question not previously addressed is whether the surrogates have any value in the more commonly used list approach. The issue is somewhat problematic, since the features used in both approaches are not identical and the use of surrogates in the list approach, as currently described, could theoretically allow some variables to be counted twice; for example, MCP joint swelling would be both an indicator of hand arthritis and a surrogate for erosion. However, our analysis of the performance of the list approach was undertaken using the surrogates, and our results showed that there were 98 additional subjects who would have been RA positive by the list at baseline (an increase in prevalence of 15%). Interestingly, of these subjects, 71 (72%) could be classified as having RA by 5 years based on the use of real data at that time point. Thus, the surrogates suggested for the tree approach have useful predictive validity for RA.

There are some limitations to be considered. As suggested above, the lower prevalence of RA at presentation using the list definition at baseline was expected because radiographs were not obtained until year 1. In contrast, the tree approach allowed for missing radiographic data, as mentioned above. Thus, at the baseline visit, classification using the list was based on only 6 variables. It was assumed that most patients would have been negative for erosions at baseline because ethical concerns were expressed at the start of the study about exposing the patients to unnecessary radiation. This assumption, however, is unlikely to be valid, although robust data from comparable patient populations are limited. The Leiden Early Arthritis Cohort comprised 524 subjects with inflammatory arthritis affecting at least 1 joint. A total of 15% of patients in that cohort had erosions at presentation (10), although unlike the current cohort, this was not based on attendance in a primary care setting. Within our cohort, erosions were likely to be less common at presentation, and some of those with erosions were likely to satisfy either ≥4 other criteria or ≤2 other criteria, and therefore, their RA status would not have been affected if a radiograph had been available. Thus, the difference in RA prevalence between the list and tree definitions is likely to be due partly to an underestimate of prevalence using the list definition, because of the missing data on erosions, and due partly to an overestimate using the tree definition, because of the use of the surrogate variable.

Second, comparing the kappa statistics over time is difficult, since the prevalence of RA (and therefore the expected agreement by chance) increases over time. Thus, the same absolute agreement at a later time will give a lower kappa statistic.

In conclusion, the 2 formulations of the ACR 1987 criteria for RA perform equally well at 5 years after presentation with IP, and the use of surrogates for missing RF and erosion data does not cause an unacceptably high level of misclassification, compared either with the use of real data or with the use of the traditional list approach. The use of the decision tree at presentation will, with its surrogate option, yield a higher initial prevalence of RA, but the arthritis identified may be milder.

Acknowledgements

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. Acknowledgements
  7. REFERENCES

We are grateful to Diane Bunn, the NOAR Clinical Manager, and the team of NOAR nurses who contributed to data collection over the period covered by this study. We also acknowledge the collaboration of the local primary care physicians as well as the rheumatologists at the Norfolk and Norwich University Hospital in the recruitment of patients.

REFERENCES

  1. Top of page
  2. Abstract
  3. PATIENTS AND METHODS
  4. RESULTS
  5. DISCUSSION
  6. Acknowledgements
  7. REFERENCES