Physician ability to assess rheumatoid arthritis disease activity using an electronic medical record–based disease activity calculator

Authors


Abstract

Objective

To assess physicians' concordance with Disease Activity Score in 28 joints (DAS28) categories calculated by an electronic medical record (EMR)–embedded disease activity calculator, as well as attitudes toward this application.

Methods

Fifteen rheumatologists used the EMR-embedded disease activity calculator to predict a rheumatoid arthritis (RA) DAS28 disease activity category at the time of each clinical encounter.

Results

Physician-predicted DAS28 disease activity categories ranged from high (>5.1, 15% of cohort, 66 of 429 patient visits) to moderate (>3.2–5.1, 21% of cohort, 90 of 429 patient visits) to low (2.6–3.2, 29% of cohort, 123 of 429 patient visits) to remission (<2.6, 35% of cohort, 150 of 429 patient visits). Overall concordance between calculated DAS28 results and physician-predicted RA disease activity was 64%. Using either the physician-predicted or the calculated DAS28 category as the gold standard, accuracy was greatest for patients in remission (75% and 88% accuracy, respectively) and those with high disease activity (68% and 79% accuracy, respectively), and less for patients with moderate (48% and 62% accuracy, respectively) or low disease activity (62% and 31% accuracy, respectively).

Conclusion

Accurate physician prediction of DAS28 remission and high disease activity categories, even without immediate availability of the erythrocyte sedimentation rate or the C-reactive protein level at the time of the visit, may be used to guide quantitatively driven outpatient RA management.

This article was published online on March 30, 2009. An error was subsequently identified in Figure 4. This notice is included in the online and print versions to indicate that both have been corrected.

INTRODUCTION

Rheumatoid arthritis (RA) is a chronic, debilitating disease, the progression of which can now be impeded by optimal treatment with newer disease-modifying antirheumatic drugs and biologic agents using quantitatively driven approaches to management of disease activity (1–4). The Disease Activity Score in 28 joints (DAS28) is a composite score derived from the 44-joint count Disease Activity Score (DAS) and has been validated to measure disease activity in clinical trials of RA (5–8). The DAS28 is calculated using the tender joint count, swollen joint count, patient global health assessment, and a serum acute-phase reactant (either the erythrocyte sedimentation rate [ESR] or the C-reactive protein [CRP] level) (9–11).

The European League Against Rheumatism (EULAR) response criteria (based on the change in DAS or DAS28) and the American College of Rheumatology (ACR) response criteria (based on the percent improvement of 7 individual clinical parameters) are the standards for assessing efficacy of drug therapy for RA in clinical trials (6, 12) and have comparable validity (8). However, serial measurement of the DAS or DAS28 may be more applicable to the routine clinical care of patients because the DAS and DAS28 each provide numeric quantification of absolute disease activity at a single time point. Thus, trends of the DAS or DAS28 can be used to monitor RA disease activity and response to therapy over time. The ACR criteria only measure relative improvement over time and have been validated only for use in randomized controlled clinical trials to distinguish active drug from placebo (12).

In the Tight Control of Rheumatoid Arthritis Study (TICORA) and Behandelstrategieën voor Reumatoide Artritis (BeSt) trials of drug therapy for RA, change in the DAS to achieve low disease activity or remission dictated whether or not treatment needed to be advanced according to protocol (1, 3). In each of these trials, tight control of RA disease activity, as measured by DAS, resulted in improved physical function and less progression of radiographic damage (1, 3).

Since the publication of these studies, physicians have begun to recognize the importance of applying the concept of tight disease control, which is used to manage other chronic diseases such as diabetes and hypertension, to the routine care of patients with RA (13). To achieve this quantitatively driven approach with the care of patients with RA, some have attempted to incorporate the use of standardized disease activity indices, such as the DAS28, into daily clinical practice. When a calculated disease activity score is used to assess and monitor disease activity in RA in clinical practice, the DAS28 is generally preferred over the DAS because the reduced number of counted joints eases the clinical work flow (7, 9). However, the routine use of the DAS28 within the context of time-limited outpatient visits must still overcome several obstacles: the time needed to gather and record the required data, the delay incurred while waiting for the result of an ESR or CRP level obtained at the time of the office visit, and the difficulty of calculating the DAS28 score given the complexity of its equation.

Several models have been proposed to integrate a standardized measure of disease activity into clinical practice in a timely fashion: a mail-in DAS28 calculating tool, in which participating physicians send patient data to a central site and then receive back by postal mail a calculated DAS score with both graphed and listed summaries of trended data (14); a DAS28 score, calculated using an ESR result obtained earlier that day by requiring the patient to come in for phlebotomy several hours before the office visit (15); a Simplified Disease Activity Index (SDAI), in which the score consists simply of the sum of each of the DAS28 components with the addition of the evaluator's global assessment of disease activity (16); and a Clinical Disease Activity Index (CDAI), in which the score is the sum of each of the DAS28 components, omitting the ESR or CRP level, with the addition of the evaluator's global assessment of disease activity (17).

We designed a disease activity calculation tool that is incorporated into our institution's electronic medical record (EMR) (18) to promote the application of quantitatively driven management of RA, as exemplified by the TICORA and BeSt studies, to the care of individual patients by facilitating DAS28 calculation in clinical practice. In this study, we assessed the concordance between calculated DAS28 scores and physician's assessment of disease activity category.

MATERIALS AND METHODS

Study design and subject eligibility.

We conducted a 12-week prospective cohort study of 15 attending rheumatologists and rheumatology fellows who cared for 385 unique patients with RA at a total of 429 outpatient visits. Eligible study physicians used an EMR-integrated disease activity calculator as part of routine care for all visits of patients with RA to document components of the DAS28 and predict the DAS28 category (18).1

Figure 1.

Data flow included clinician entry of swollen/tender joint counts and patient global health score, ordering of C-reactive protein (CRP) level and/or erythrocyte sedimentation rate (ESR), daily automated offline calculation of the Disease Activity Score in 28 joints (DAS28), and weekly automated e-mail report to clinician. EMR = electronic medical record; ROC = Rheumatology OnCall application.

All attending rheumatologists and fellows providing at least 1 half-day session of outpatient clinical care per week at the Massachusetts General Hospital or the Brigham and Women's Hospital were eligible for enrollment in the study.

Data collection.

During routine outpatient visits, participating physicians accessed the graphical joint examination–recording tool and disease activity calculator through the EMR to enter the swollen and tender joint counts and the patient-reported general health visual analog scale (VAS) data for patients with RA. After entering these data, but before having access to the ESR or CRP laboratory result from that visit, study physicians were asked to predict the DAS28 range as high (>5.1), moderate (>3.2–5.1), low (2.6–3.2), or remission (<2.6). Thus, physicians were blinded to the results of the ESR and CRP from that visit at the time of DAS28 disease activity category prediction. If not obtained within the previous 7 days, study physicians were instructed to order ESR and CRP level tests at the time of the outpatient visit. DAS28 scores were then automatically calculated the following day for all patients for whom joint count and global health VAS data were entered and for whom an ESR or CRP result from within the previous 7 days was available.

Statistical analysis.

Data were summarized as proportions or means with SDs. We specifically tested the hypothesis that using chi-square tests, concordance between physician-predicted disease activity and calculated DAS28 scores would be greater at the extremes of the disease spectrum (high and remission disease activity states) versus those in the middle of the disease activity spectrum (moderate and low disease activity states). We also examined concordance rates comparing more experienced versus less experienced physicians, and comparing physicians with more versus fewer clinical sessions per week. Comparison groups were chosen a priori based on face-valid thresholds (i.e., 10 years of experience, half-time clinical practice) and to divide respondents into 2 roughly equal groups (e.g., 7 years of experience, 3 or more sessions per week). We used generalized estimating equations (PROC GENMOD, SAS version 9.1; SAS Institute, Cary, NC) to account for patient clustering by physician for all statistical comparisons.

RESULTS

Study population.

Of 47 clinically active attending rheumatologists and fellows at Massachusetts General Hospital and Brigham and Women's Hospital, 25 agreed to participate in the study. Five of the invited participants declined, stating they do not regularly see patients with RA. The remaining 17, including 13 attending physicians and 4 fellows, did not respond to the invitation. Fifteen (9 attendings and 6 fellows) of the 25 rheumatologists who agreed to participate used the disease activity calculator at least once over the course of the 12-week study. The 15 study physicians practiced during a mean of 3 half-day clinical sessions per week (range 1–7), and they had engaged in the practice of rheumatology for a mean of 9 years (range 1–28).

Data capture.

Over the course of the 12-week study, the disease activity calculator was used to document a joint examination and general health VAS during 474 patient visits (Figure 2). A DAS28 prediction was made during 429 of these visits, and an ESR or CRP level was subsequently available to calculate a DAS28 score after 398 visits.

Figure 2.

Data capture from 25 enrolled rheumatologists yielded 429 Disease Activity Score in 28 joints (DAS28) predictions and 398 calculated DAS28 scores. GH = global health; VAS = visual analog scale; ESR = erythrocyte sedimentation rate; CRP = C-reactive protein.

DAS28 prediction.

Physician-predicted DAS28 categories ranged from high disease activity (DAS28 >5.1) in 66 of 429 patient visits, or 15% of the cohort, to remission (DAS28 <2.6) in 150 of 429 patient visits, or 35% of the cohort. The frequency of calculated DAS28 scores for each category ranged from high disease activity (65 of 385 patient visits, or 17% of the cohort) to remission (160 of 385 patient visits, or 42% of the cohort) (Figure 3). Overall concordance between calculated DAS28 results, grouped by disease activity category, and physician-predicted disease activity categories was 64% (248 of 385 patient visits).

Figure 3.

Frequency of physician (MD) Disease Activity Score in 28 joints (DAS28) category predictions and calculated DAS28 scores. Dark bars represent MD predictions (n = 429) and light bars represent DAS28 calculations (n = 398).

Physicians most accurately predicted DAS28 remission (120 of 137 patient visits, 88% accuracy) or high disease activity (44 of 56 patient visits, 79% accuracy). They less accurately predicted moderate disease activity (50 of 81 patient visits, 62% accuracy) and least accurately predicted low disease activity (34 of 111 patient visits, 31% accuracy) (Figure 4A). Similarly, the categorized DAS28-calculated results concurred most with physician-predicted DAS28 categories in patients with predicted remission (120 of 160 patient visits, 75% accuracy) or with high disease activity (41 of 61 patient visits, 68% accuracy), and least in patients with moderate (50 of 104 patient visits, 48% accuracy) or low disease activity (34 of 55 patient visits, 62% accuracy) (Figure 4B).

Figure 4.

Concordance of physician (MD) Disease Activity Score in 28 joints (DAS28) predictions and calculated DAS28 scores, A, using the physician-predicted DAS28 category as the gold standard, and B, using the calculated DSA28 as the gold standard.

The concordance between physician-predicted and calculated DAS28 ranges was significantly higher in the combined high disease activity and remission ranges (164 [85%] of 193 patient visits) than in the low and moderate disease activity ranges (84 [44%] of 192 patient visits; P ≤ 0.001). The concordance between physician-predicted and calculated DAS28 ranges was significantly lower in the low disease activity range (34 [31%] of 111 patient visits) than in the combined high and moderate disease activity and remission ranges (214 [78%] of 274 patient visits; P ≤ 0.001).

Of all inaccurate physician predictions, under-predictions (76 [55%] of 137 patient visits) (predicting a DAS28 range lower than the calculated DAS28) occurred slightly more frequently than over-predictions (61 [45%] of 137 patient visits). The frequency of inaccurate physician predictions within the group of calculated low DAS28 scores was essentially equal between over-predictions (38 [49%] of 77 patient visits) and under-predictions (39 [51%] of 77 patient visits). Inaccurate physician predictions were off by more than 1 disease activity category (remission, low, moderate, high) in only 9% of predictions (129 of 137 patient visits) (Figure 4).

In the group of all discordant calculated DAS28 results, grouped by category and physician-predicted disease activity categories (137 of 385 patient visits), none of the 3 variables (joint count, patient global health assessment, or acute-phase reactant result) was discordantly high or low when compared with the other variables in 55 (40%) of 137 encounters. In the remaining 82 encounters for which a single variable was discordant with the other variables, that variable was the joint count in 28 (20%) of 137 encounters, the patient global health assessment in 27 (20%) of 137 encounters, and the acute-phase reactant result in 27 (20%) of 137 encounters.

There was no statistically significant difference in the overall concordance of predictions between physicians grouped by years of experience (P > 1.0 using either 7 or 10 years in practice as thresholds for comparison) or by number of sessions per week (P > 1.0 using either 3 or 4 sessions/week as thresholds for comparison).

DISCUSSION

Although a DAS28 cannot replace clinical judgment in the context of the individual patient visit, quantitatively driven management of RA based on absolute values and changes in the DAS or DAS28 has been shown to improve patient outcomes in randomized controlled clinical trials (1, 2, 4). Therefore, it is valuable for the clinician to supplement and influence his or her subjective clinical judgment with the DAS28, and by going through the process of recording the data used to calculate the DAS28 at the time of the visit.

In considering the results of this study, one must ask what the gold standard for disease activity in RA is: the calculated DAS28 or the physician's assessment of the disease activity category? This is actually a circular argument because the original DAS disease activity categories, on which the DAS28 disease activity categories are based, were developed by calculating the DAS for patients grouped into categories of high and low disease activity based on their physicians' treatment decisions at the time of the clinical encounter (6). To limit the overlap between high and low disease activity categories to ≤5%, the high disease activity category was defined as DAS results above the 25th percentile of DAS calculated for patients classified as having high disease activity, and the low disease activity category was defined as DAS results at or below the 75th percentile of DAS calculated for patients classified by physicians as having low disease activity. Using these cutoffs, the high (>3.7), moderate (>2.4 and ≤3.7), and low (≤2.4) categories of disease activity became accepted standards for assessment of treatment response in clinical trials as defined in the EULAR response criteria. The DAS28, CDAI, and SDAI were then validated by assessing agreement between each tool and the previously developed iterations of the DAS, as well as the Health Assessment Questionnaire and the ACR response criteria (12).

Regardless of whether the calculated DAS28 or the physician-predicted disease activity category is considered to be the gold standard, rates of concordance between calculated DAS28 and physician-predicted disease activity categories in this study were similar for each category (Figure 4). Although the overall concordance between physician-predicted and calculated DAS28 categories was only 65%, the concordance between physician-predicted and calculated individual categories was greatest at the extremes of disease activity (high disease activity and remission). This suggests that clinical trials of quantitatively driven management of RA should use only the most extreme DAS28 categories (high and remission) as primary end points because these are the categories most accurately predicted by physicians. The Combination of Methotrexate and Etanercept in Active Early Rheumatoid Arthritis study was the first quantitatively driven clinical trial of RA therapy to use DAS28 remission as a primary end point (19).

The regular use of a disease activity calculator has advantages aside from confirming the physician's clinical impression. Documentation of the individual components of the DAS28 focuses the physician's attention on the detailed joint examination and on the patient's reported assessment of global health. In this way, a disease activity calculator incorporated into the EMR is useful to guide outpatient RA care, even without the immediate availability of ESR or CRP at the time of the visit. In addition, use of the disease activity calculator allows numeric quantification and documentation of the physician's overall clinical impression, as encompassed in a DAS28 score, and encourages physicians to strive for and achieve tighter control of RA disease activity by complying with DAS28-guided medication adjustments, as has been shown to be beneficial in clinical trials (2, 3).

The major factor that limits the use of our disease activity calculator to calculate DAS28 at the point of service in clinical care is the unavailability of a recent acute-phase reactant determination at the time of the outpatient encounter. Testing for acute-phase reactants typically occurs at the time of the encounter, rather than before the outpatient visit. This delays calculation of the DAS28 until several hours after phlebotomy has been performed. Thus, treatment decisions that are based on the DAS28 score itself cannot be made at the time of the visit. This limitation, however, may be balanced by the experience, reported by physicians participating in this study, that the act of recording the data elements used to calculate the DAS28 and the availability of graphically trended disease activity scores themselves improve the care of patients with RA (18).

Aside from the logistical issues associated with using the DAS28 in clinical practice, the DAS28 disease activity categories might not be an accurate measure of absolute disease activity for all members of a heterogeneous population of patients with RA and various comorbidities, or for patients with pain amplification syndromes. A recent study comparing the calculated DAS28 with the physician's estimate of global RA disease activity at 669 outpatient visits reported routinely higher assessment of disease activity using the DAS28 calculation as compared with the physician's estimate (20). However, the graphic presentation in our EMR-integrated platform of DAS28 trends over time for individual patients allows physicians to recognize changes in disease activity independently of disease activity categories.

Although the DAS28 was not calculated and made available to the rheumatologist until the morning after the patient visit, most physicians in this study cohort reported that use of the disease activity calculator improved outpatient care of patients with RA (18). Therefore, this physician perception that patient care is improved must be related to the physician's attention being focused on either or both the detailed joint assessment and the patient's assessment of his or her global health. The duration of clinical experience or the frequency of regular outpatient clinical activity made little difference in the accuracy of physicians' prediction of DAS28 in subset analyses. Thus, incorporation of an EMR-based disease activity calculator into the clinical work flow should be useful to physicians at all stages of clinical practice.

The use of a disease activity calculator incorporated into the EMR improves patient care by consistently requiring the clinician to perform a detailed joint examination, by focusing the clinician's attention directly on the patient's assessment of his or her global health, and by quantitating disease activity at each patient visit. Use of this tool will facilitate the adjustment of medical therapy of RA, according to quantitatively driven management protocols, so as to achieve tighter and better disease control. Although the CDAI has not yet become a standard measure of disease activity in clinical trials of quantitatively driven management of RA, future versions of our EMR-based disease activity calculator may incorporate a VAS of the physician global assessment of disease activity to automate calculation of the CDAI at the time of the outpatient encounter. This will allow real-time monitoring of RA disease activity and will guide changes in therapy at the time of the patient encounter, without the need to wait for results of laboratory testing for an acute-phase reactant.

AUTHOR CONTRIBUTIONS

Dr. Collier had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study design. Collier, Grant, Estey, Surrao, Chueh, Kay.

Acquisition of data. Collier, Kay.

Analysis and interpretation of data. Collier, Grant, Estey, Surrao, Kay.

Manuscript preparation. Collier, Grant, Kay.

Statistical analysis. Collier, Grant, Kay.

Ancillary