Classification of rheumatoid arthritis: Comparison of the 1987 American College of Rheumatology criteria and the 2010 American College of Rheumatology/European League Against Rheumatism criteria

Abstract

Objective

New criteria to classify rheumatoid arthritis (RA) have been derived in order to increase the specificity and sensitivity for early RA compared with the 1987 American College of Rheumatology (ACR) criteria. The aim of this study was to evaluate differences in classification between the 1987 ACR criteria and the 2010 ACR/European League Against Rheumatism (EULAR) criteria in an early arthritis cohort and to determine the test characteristics of the 2010 ACR/EULAR criteria.

Methods

A total of 2,258 patients with early arthritis included in the Leiden Early Arthritis Clinic cohort were studied. Fulfillment of the 1987 and 2010 criteria for the classification of RA was determined at baseline. The diagnosis of each patient at 1 year was assessed. The sensitivity and specificity of the 2010 criteria were determined using the following outcome measures: initiation of methotrexate therapy or any disease-modifying antirheumatic drug (DMARD) therapy during the first year of followup and having persistent arthritis during 5 years of followup.

Results

At their first presentation, 1,099 patients fulfilled the 2010 criteria, and 726 patients fulfilled the 1987 criteria for RA. Eighty-two of the 726 patients fulfilling the 1987 criteria did not fulfill the 2010 criteria. Sixty-eight percent of the patients who fulfilled the 1987 criteria during the first year of disease but not at baseline did fulfill the 2010 criteria at baseline. In 18% of patients, use of the 2010 classification criteria also led to a revoked classification at 1 year. The sensitivity and specificity of the 2010 criteria were 0.84 and 0.60, respectively, with methotrexate therapy as the outcome and 0.74 and 0.74, respectively, with DMARD therapy as the outcome. With persistent arthritis as the outcome, the sensitivity and specificity of the 2010 criteria were 0.71 and 0.65, respectively.

Conclusion

Compared with the 1987 criteria, the 2010 criteria classify more patients with RA and at an earlier phase of the disease. The discriminative ability of the 2010 criteria is acceptable.

During the last decades, the focus of the management of rheumatoid arthritis (RA) has shifted to the early phase of the disease. This change was fueled by studies showing that early achievement of a low disease activity state is beneficial for the further course of RA. These studies raised awareness regarding the importance of early treatment and pointed to the relevance of early recognition of RA. From this perspective, the 1987 American College of Rheumatology (ACR) criteria for RA (1) have been criticized, because they are not equipped to diagnose early RA. This is not surprising, because the 1987 criteria were developed in order to define homogeneous patient groups for research purposes and therefore were based on patients in whom the average disease duration was 7 years.

In order to be able to identify patients with early RA for clinical trials and other studies, new classification criteria for RA were derived by a task force of experts from both the European League Against Rheumatism (EULAR) and the ACR (2). The main purpose of these 2010 criteria is to achieve increased sensitivity and specificity for identifying RA in an early phase.

At present, the diagnostic and discriminative abilities of these new criteria are not known. In addition, it is unclear how the 2010 ACR/EULAR criteria perform in relation to the 1987 ACR criteria. This is especially relevant, because the working group that developed the 2010 criteria stressed in their discussion that patients fulfilling the 2010 criteria are probably less homogeneous and, therefore, that in clinical trials, researchers should “document the proportions of study subjects who fulfill the previous (1987) criteria and the new RA classification criteria, to enable comparisons.” Moreover, the working group warned that the 2010 criteria may increase heterogeneity by including different phenotypes, thereby making basic science studies more difficult.

Therefore, the present study evaluated the following questions: What proportion of patients with early arthritis who do not fulfill the 1987 criteria can, according to the 2010 criteria, be classified as having RA? Do all patients with early arthritis who fulfill the 1987 criteria fulfill the 2010 criteria as well? Do patients with RA indeed fulfill the 2010 criteria at an earlier point in time than they would have using the 1987 criteria? In addition, the sensitivity and specificity of the 2010 criteria for RA were assessed. For this analysis, 3 outcome measures were studied: initiation of methotrexate (MTX), initiation of any disease-modifying antirheumatic drug (DMARD), and having persistent arthritis over a 5-year followup period.

PATIENTS AND METHODS

Patients.

The study group comprised patients with early arthritis from the Leiden Early Arthritis Clinic (EAC) cohort, a population-based prospective cohort that started in 1993 (3). Inclusion in the study took place when arthritis was confirmed by physical examination and the symptom duration was <2 years. The inclusion criteria had not changed over time. At the first visit, patients and rheumatologists completed questionnaires, a physical examination was performed, radiographs were obtained, and blood was obtained for determination of the C-reactive protein (CRP) level, the erythrocyte sedimentation rate (ESR), and the presence of IgM rheumatoid factor (RF) and anti–citrullinated protein antibodies (tested as anti–cyclic citrullinated peptide [anti-CCP2], Immunoscan RA Mark 2; Euro-Diagnostica). Followup visits were performed yearly. Written informed consent was obtained from all participants. The study was approved by the local medical ethics committee.

A total of 2,258 patients with early arthritis were included between 1993 and February 2009, and these patients were followed up for at least 1 year. Of these patients, 1,632 were formerly classified as having RA (according to the 1987 ACR criteria) or undifferentiated arthritis. The remaining 626 patients with early arthritis were classified as having other diagnoses. The treatment of patients with RA differed; hydroxychloroquine, penicillamine, or sulfasalazine were the initial DMARDs in the 1990s, and MTX was the initial DMARD since 1999 (4). Patients who were classified as having diagnoses other than RA were treated accordingly. No protocol was established for the treatment of patients who were classified as having undifferentiated arthritis.

Application of the 2010 criteria.

The 2010 ACR/EULAR criteria were applied as described by Aletaha et al (2). We used the swollen joint count in 66 joints and the tender joint count in 68 joints. According to the guidelines, the distal phalangeal joints, first carpometacarpal joint, and first metatarsophalangeal joints were excluded from assessment. Involvement of the interphalangeal joints of the feet was considered to be small-joint involvement. The reference value for RF positivity in our cohort was 5 AU; therefore, a level ≥15 AU was considered to be high-level positive. Similarly, the reference value for anti-CCP2 positivity in our cohort was 25 AU, and a level of ≥75 AU was considered to be high-level positive. An abnormal CRP level was defined according to the reference value of ≥10 mg/liter, and an abnormal ESR was defined as ≥25 mm/hour for women and ≥15 mm/hour for men.

According to the new criteria, the presence of a significant erosion is prima facie evidence of RA that precludes the need for applying other criteria. However, there currently is no agreement regarding the size, number, or site of erosions necessary to define erosive disease. Because of this uncertainty, we initially did not consider radiographic information when applying the 2010 ACR/EULAR criteria. Afterward, analyses were repeated when patients with a score ≥2 for erosiveness according to the Sharp/van der Heijde system (5) were classified as having RA as well, irrespective of fulfilling any of the other criteria. In addition, the effect of evaluating 44 or 28 joints instead of 66 and 68 joints was assessed.

Analyses.

The following analyses were performed. First, the baseline characteristics of all 2,258 patients with early arthritis were studied to define the proportions of patients who were classified as having RA according to the 2010 criteria and the 1987 ACR criteria. It was also determined whether patients with a diagnosis of RA according to the 1987 ACR criteria fulfilled the 2010 criteria as well.

In order to determine whether the 2010 criteria identify disease at an earlier stage than that identified by the 1987 criteria, 297 patients who did fulfill the 1987 criteria during the first year of disease but not at their first presentation were studied. It was determined how many of these patients already fulfilled the 2010 ACR criteria at baseline and thus indeed had disease that was recognized at an earlier phase by the 2010 criteria.

The question of whether the 2010 criteria would yield false-positive classifications was also evaluated. To this end, patients who at baseline fulfilled the 2010 ACR criteria were studied for their diagnosis after 1 year, to determine whether they were classified differently at the later time point.

Finally, the sensitivity and specificity of the 2010 criteria were determined, and the area under the curve (AUC) was assessed in the patients who were formerly classified as having RA or undifferentiated arthritis (n = 1,632). Patients who received DMARD treatment in a randomized trial were not studied, leaving 1,404 patients who were treated by their rheumatologists for evaluation. Three outcome measures were used. The first outcome measure was the initiation of MTX therapy within the first year of followup; this outcome measure is the same as that used for derivation of the 2010 criteria. Because MTX was not the anchor drug before 1999, initiation of any DMARD within the first year was used as the second outcome measure. The third outcome measure, persistence of arthritis, defined as the absence of a sustained DMARD-free remission, was assessed in the subgroup of patients who achieved 5 years of followup (n = 790). Patients were classified as having disease in remission if DMARD therapy could be discontinued and no synovitis was detected for at least 1 year (6). The analyses were performed using SPSS version 17.0 software.

RESULTS

The baseline characteristics of all patients with early arthritis are presented in Table 1. The characteristics of the subsets of patients who at baseline were classified as having RA according to the 1987 ACR criteria and the 2010 ACR/EULAR criteria are also shown.

Table 1. Baseline characteristics of all patients with early arthritis, the subset of patients who fulfilled the 1987 ACR criteria at first presentation, and the subset of patients who fulfilled the 2010 ACR/EULAR criteria at first presentation*

Characteristic	All patients with early arthritis (n = 2,258)	Patients who fulfilled the 1987 criteria (n = 726)	Patients who fulfilled the 2010 criteria (n = 1,099)
* Except where indicated otherwise, values are the number (%) of patients. The 2010 criteria were applied without considering data on erosiveness at baseline. In the group of all patients with early arthritis, some data were missing, as follows: for symptom duration, n = 220; for erythrocyte sedimentation rate (ESR), n = 15; for C-reactive protein (CRP), n = 219; for rheumatoid factor (RF), n = 32; for anti–cyclic citrullinated peptide (anti–CCP-2), n = 553. † The criteria were subdivided according to score, based on the algorithm from the 2010 American College of Rheumatology/European League Against Rheumatism (ACR/EULAR) criteria. ‡ According to the 1987 criteria, 4 of 7 criteria must be present for >6 weeks.
Age at inclusion, mean ± SD years	51.9 ± 17.2	57.4 ± 16.3	56.1 ± 16.4
Female sex	1,340 (59.3)	470 (64.7)	718 (65.3)
Symptom duration at first presentation, mean ± SD weeks	25.9 (41.6)	31.6 (36.3)	29.8 (43.4)
<6 weeks†	436 (21.4)	0 (0)‡	98 (9.6)
≥6 weeks†	1,602 (78.6)	726 (100)‡	925 (90.4)
Swollen joint count of 66 joints, mean ± SD†	6.5 ± 6.8	11.6 ± 7.3	10.3 ± 7.7
1 medium/large joint	253 (11.2)	0 (0)	0 (0)
2–10 medium/large joints	142 (6.3)	0 (0)	3 (0.3)
1–3 small joints	532 (23.6)	48 (6.6)	106 (9.6)
4–10 small joints	560 (24.8)	206 (28.4)	236 (21.5)
>10 joints	771 (34.1)	472 (65.0)	754 (68.6)
ESR, mean ± SD mm/hour	33.2 ± 28.1	40.3 ± 28.2	38.3 ± 28.0
CRP, mean ± SD mg/liter	27.1 (28.1)	28.2 (35.2)	28.3 (35.1)
Normal CRP and normal ESR†	747 (33.1)	147 (20.2)	251 (22.8)
Abnormal CRP or abnormal ESR†	1,511 (66.9)	579 (79.8)	848 (77.2)
RF positive	671 (30.1)	399 (55.0)	601 (55.3)
Anti–CCP-2 positive	506 (29.7)	323 (51.4)	472 (52.2)
Negative RF and negative anti–CCP-2†	1,484 (65.7)	285 (39.3)	412 (37.5)
Low-positive RF or low-positive anti–CCP-2†	203 (9.0)	78 (10.7)	145 (13.2)
High-positive RF or high-positive anti–CCP-2†	571 (25.3)	363 (50.0)	542 (49.3)
Erosive disease	590 (26.1)	392 (54.0)	467 (42.5)

Agreement in classification.

At baseline, 1,099 of 2,258 patients with early arthritis fulfilled the 2010 ACR/EULAR criteria for RA, and 726 patients fulfilled the 1987 ACR criteria for RA. Among these 726 patients, 644 (88.7%) also fulfilled the 2010 criteria, whereas 82 patients (11.3%) did not fulfill the 2010 criteria. Among the 1,099 patients who fulfilled the 2010 criteria, 455 patients did not fulfill the 1987 criteria (Table 2). The agreement in classification criteria was not different when patients included before or after 1999 were studied separately (data not shown). Supplementary Table 1 (available in the online version of this article at http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1529-0131) shows the characteristics of the patients who fulfilled both the 1987 and 2010 criteria and the patients who fulfilled the 1987 criteria but not the 2010 criteria.

Table 2. Classification of RA according to the 1987 and 2010 criteria, without and with radiography information on erosiveness when applying the 2010 ACR/EULAR criteria*

1987 ACR classification criteria	2010 ACR/EULAR classification criteria
	Without radiography information		With radiography information
	RA at baseline	No RA at baseline	RA at baseline	No RA at baseline	Total
* Criteria were applied using the swollen joint count in 66 joints and the tender joint count in 68 joints. When using the 2010 American College of Rheumatology/European League Against Rheumatism (ACR/EULAR) criteria with radiography information, erosiveness was defined as a total Sharp/van der Heijde erosion score ≥2. RA = rheumatoid arthritis.
RA at baseline	644	82	678	48	726
No RA at baseline	455	1,077	544	988	1,532
Total	1,099	1,159	1,222	1,036	2,258

Baseline classification in relation to the disease course.

A total of 297 patients fulfilled the 1987 ACR criteria during the first year but not at baseline. Among these patients, 202 (68.0%) did fulfill the 2010 criteria at baseline, indicating that the 2010 criteria did indeed classify RA at an earlier phase of the disease.

The 1,099 patients with early arthritis who fulfilled the 2010 ACR/EULAR criteria at baseline were evaluated for their diagnosis at 1 year. One hundred ninety-four patients were classified differently at that time point. A review of the medical records of these patients confirmed that they had clinical evidence of a diagnosis other than RA. The diagnoses of these patients were psoriatic arthritis (n = 46), inflammatory osteoarthritis (n = 28), reactive arthritis (n = 20), remitting seronegative symmetrical synovitis with pitting edema (n = 17), sarcoidosis (n = 15), (pseudo)gout (n = 15), paramalignant arthritis (n = 6), spondylarthropathy (n = 6), systemic lupus erythematosus (n = 10), mixed connective tissue disease (3), other systemic disorders (n = 21), and other diagnoses (n = 7). These patients comprised 17.7% of the total population of patients fulfilling the 2010 ACR/EULAR criteria and 27.7% of the patients who at baseline fulfilled the 2010 criteria but not the 1987 criteria.

In their description of the 2010 criteria for RA, Aletaha and colleagues state that these criteria should be applied only when no other diagnosis can be established. Thus, when a patient can be classified as having 2 diagnoses, the diagnosis other than RA prevails. Therefore, we repeated the analysis described above in the patients who were formerly classified as having RA or undifferentiated arthritis (n = 1,632). Of these, 939 patients fulfilled the 2010 ACR/EULAR criteria at baseline and were evaluated for their diagnosis at 1 year. Eighty-eight patients were classified differently at that time point. A review of the medical records of these patients confirmed that they had clinical evidence of a diagnosis other than RA. The diagnoses of these patients were psoriatic arthritis (n = 20), inflammatory osteoarthritis (n = 13), reactive arthritis (n = 7), remitting seronegative symmetrical synovitis with pitting edema (n = 7), (pseudo)gout (n = 7), systemic lupus erythematosus (n = 6), paramalignant arthritis (n = 4), spondylarthropathy (n = 4), sarcoidosis (n = 3), mixed connective tissue disease (n = 2), other systemic disorders (n = 12), and other diagnoses (n = 3). These patients comprised 9.4% of the 939 patients who fulfilled the 2010 ACR/EULAR criteria and 14.1% of the patients who at baseline fulfilled the 2010 criteria but not the 1987 criteria.

Test characteristics of the 2010 criteria.

When using initiation of MTX therapy within the first year as the outcome, the sensitivity and specificity of the 2010 criteria were 0.84 and 0.60, respectively. When using initiation of any DMARD therapy within the first year as the outcome, the sensitivity and specificity were 0.74 and 0.74, respectively. When using these 2 outcomes, the AUC values were 0.72 and 0.74, respectively. When using the third outcome measure, persistence of arthritis over 5 years, the sensitivity of the 2010 criteria was 0.71, the specificity was 0.65, and the AUC value was 0.65 (Table 3).

Table 3. Test characteristics of the 1987 and 2010 criteria using 3 outcome measures, based on analyses performed without and with radiography information on erosiveness when applying the 2010 ACR/EULAR criteria*

Outcome measure	Classification criteria
	1987 ACR	2010 ACR/EULAR
		Without radiography information	With radiography information
* When using the 2010 American College of Rheumatology/European League Against Rheumatism (ACR/EULAR) criteria with radiography information, erosiveness was defined as a total Sharp/van der Heijde erosion score ≥2. MTX = methotrexate; AUC = area under the curve; DMARD = disease-modifying antirheumatic drug.
MTX intiation
Senstivity	0.61	0.84	0.88
Specificity	0.74	0.60	0.54
AUC	0.67	0.72	0.71
DMARD initiation
Senstivity	0.54	0.74	0.79
Specificity	0.87	0.74	0.68
AUC	0.71	0.74	0.79
Persistent disease for 5 years
Senstivity	0.53	0.71	0.77
Specificity	0.75	0.65	0.56
AUC	0.61	0.65	0.65

Benefit of including erosiveness.

It is unclear what number of erosions in patients with early arthritis is specific for early RA. When evaluating the consequences of considering erosiveness, herein defined as a total Sharp/van der Heijde erosion score ≥2, we observed that at baseline, 1,222 patients were classified as having RA according to the 2010 criteria. When erosiveness was included in the evaluation, 123 patients with early arthritis (5.4%) were additionally classified as having RA. The analyses on the agreement in classification and on the test characteristics were repeated with radiography information on erosiveness included (Tables 2 and 3), but the results were not substantially different compared with the results achieved when radiographic information was disregarded.

Effect of the number of assessed joints.

We used a swollen joint count in 66 joints and a tender joint count in 68 joints. In clinical practice, swelling and tenderness in 44 joints or 28 joints may be evaluated more frequently. In order to determine whether different joint counts would result in different classifications, the 2010 criteria were applied using a 44-joint count and a 28-joint count. The numbers of patients classified as having RA were then 1,082 and 940, respectively, instead of 1,099 (see Supplementary Table 2, available in the on-line version of this article at http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1529-0131). The test characteristics were fairly comparable when a lower number of joints was considered (see Supplementary Table 3, available in the on-line version of this article at http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1529-0131).

DISCUSSION

The present study compared the classification of RA using the 1987 ACR criteria and the 2010 ACR/EULAR criteria in a large cohort of patients with early arthritis. Use of the 2010 criteria classified more patients with RA than the 1987 criteria, and 11.3% of the patients with RA according to the 1987 criteria were not classified as having RA according to the 2010 criteria. A large proportion of the patients with early arthritis who developed RA according to the 1987 criteria during the disease course could, at first presentation, be classified as having RA according to the 2010 criteria. This denotes that the 2010 criteria have met the demand of classifying RA at an earlier phase of the disease compared with the 1987 criteria. Application of the 2010 classification also led to a revoked diagnosis at a later phase in 18% of patients (or 9.5%, depending on the studied population), substantiating the concerns with regard to an increase in heterogeneity with use of the 2010 criteria. Compared with the 1987 criteria, the sensitivity of the 2010 criteria was higher, but the specificity was lower.

In this study, several choices were made when applying the 2010 ACR/EULAR criteria. Initially, we omitted information on hand and feet radiographs, because a clear description of erosive disease resembling RA was not provided by Aletaha et al (2). We then repeated the analyses, defining a total Sharp/van der Heijde erosion score ≥2 as erosiveness. Fairly similar results were observed. This may suggest that evaluating radiographs in the early phase of arthritis is not highly relevant for the classification of RA. The number of erosive joints in patients with early arthritis was recently shown to be moderately predictive of the development of RA (7).

We used a swollen joint count in 66 joints and a tender joint count in 68 joints. In clinical practice, a 44- or 28-joint count is frequently used. To evaluate the effect of assessing different numbers of joints, we repeated the analyses with the lower joint counts. The number of patients classified as having RA decreased, but the test characteristics were only marginally affected.

This study has some limitations. First, it is based on a single inception cohort, and more studies are needed to establish the sensitivity and specificity of the new criteria. A complicating factor is the ambiguity regarding which outcome measure should be used as the gold standard in RA. This was subject to discussion within the working group that derived the 2010 criteria, and the outcome measure of initiation of MTX was chosen. This outcome may not be appropriate when studying chronologically older cohorts. For example, in the 1990s it was not common practice to start MTX early in a patient with arthritis of recent onset who did not fulfill the 1987 criteria. For this reason, we chose any DMARD therapy instead of MTX therapy as an outcome. For verification, we additionally used the persistence of arthritis over 5 years as an outcome. However, differences in the outcome measures may yield variations in the observed test characteristics.

All 3 of the outcome measures used here (initiation of MTX, initiation of any DMARD, and persistence of arthritis) are associated with the risk of misclassification, because these outcome measures can also be fulfilled in patients with other diagnoses, such as psoriatic arthritis. A diagnosis of psoriatic arthritis was also the most frequent cause of a revoked classification of RA at 1 year.

Another consideration is that 213 patients with early arthritis who were included in the EAC cohort after 2000 were used in the data-driven phase of development of the 2010 criteria (8). In the present study, we evaluated a considerably larger number of patients as well as 2 outcome measures in addition to MTX treatment. In order to determine whether this subset of patients affected the results, repeat analyses were performed in which these 213 patients were excluded. This repeat analysis did not yield substantially different results (data not shown). Nonetheless, validation of the 2010 criteria in other cohorts is required.

Given the emerging evidence on a “window of opportunity” (9) pointing to the need to provide treatment as early as possible, the question arises regarding what method best identifies individual patients with RA in an early phase. The investigators who developed the 2010 criteria emphasize that the new classification criteria were not developed as a diagnostic tool, and that a separate body of work is needed to develop such tools (2). Prediction rules aimed at early diagnosis have been developed and validated (10, 11). The question of what method is best to identify early RA on the individual patient level is still unanswered and is a subject for future studies.

In conclusion, the 2010 criteria for RA classify more patients with RA and do so at an earlier phase of disease. The discriminative ability of the 2010 ACR/EULAR criteria is reasonable, indicating that these criteria perform well to classify early RA.

AUTHOR CONTRIBUTIONS

All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Dr. van der Linden had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study conception and design. Van der Linden, Knevel, Huizinga, van der Helm-van Mil.

Acquisition of data. Van der Linden, Knevel, van der Helm-van Mil.

Analysis and interpretation of data. Van der Linden, Knevel, Huizinga, van der Helm-van Mil.