SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. ROLE OF THE STUDY SPONSOR
  9. REFERENCES

Objective

Clinical trials in psoriasis and psoriatic arthritis (PsA) involve assessment of the skin and joints. This study aimed to determine whether assessment of the skin and joints in patients with PsA by rheumatologists and dermatologists is reproducible.

Methods

Ten rheumatologists and 9 dermatologists from 7 countries met for a combined physical examination exercise to assess 20 PsA patients (11 men, mean age 51 years, mean PsA duration 11 years). Each physician assessed 10 patients according to a modified Latin square design that enabled the assessment of patient, assessor, and order effect. Tender joint count (TJC), swollen joint count (SJC), dactylitis, physician's global assessment (PGA) of PsA disease activity (PGA-PsA), psoriasis body surface area (BSA), Psoriasis Area and Severity Index (PASI), Lattice System Physician's Global Assessment of psoriasis (LS-PGA), National Psoriasis Foundation Psoriasis Score (NPF-PS), modified Nail Psoriasis Severity Index (mNAPSI), number of fingernails with nail changes (NN), and PGA of psoriasis activity (PGA-Ps) were assessed. Variance components analyses were carried out to estimate the intraclass correlation coefficient (ICC), adjusted for the order of measurements.

Results

There is excellent agreement (ICC ≥0.80) on the mNAPSI, substantial agreement (0.6 ≥ ICC < 0.80) on the TJC, PASI, and NN, moderate agreement (0.4 ≥ ICC < 0.60) on the PGA-Ps, LS-PGA, NPF-PS, and BSA, and fair agreement (0.2 ≥ ICC < 0.40) on the SJC, dactylitis, and PGA-PsA. The only measure that showed a significant difference between dermatologists and rheumatologists was dactylitis (P = 0.0005).

Conclusion

There is substantial to excellent agreement on the TJC, PASI, NN, and mNAPSI among rheumatologists and dermatologists.


INTRODUCTION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. ROLE OF THE STUDY SPONSOR
  9. REFERENCES

Psoriasis is a chronic, immune-mediated, inflammatory skin disease affecting at least 2.2% of the general population in North America (1). Approximately 30% of patients with psoriasis have psoriatic arthritis (PsA) (2). Comprehensive assessment of patients with psoriasis and PsA involves assessment of the skin, nails, joints, and dactylitis (3–7).

To assess skin involvement, the most commonly used measure, especially in clinical trials, is the Psoriasis Area and Severity Index (PASI), which scores the average redness, thickness, and scaliness of the lesions (0–4 scale), weighted by the area of involvement. PASI scores range from 0 to 72, where higher scores indicate more severe disease (8). Another frequently used scale is the physician's global assessment (PGA) of psoriasis activity (PGA-Ps), rated on a scale from clear or inactive to severe or most active (5). Although more subjective, this scale more closely resembles the assessment used by dermatologists in clinical practice (9). The Lattice System Physician's Global Assessment (LS-PGA) is a relatively new tool to quantify psoriasis severity that provides an 8-step result from clear to very severe (10). The LS-PGA takes a quantitative approach to global assessment by integrating the ranges of the percentage of body surface area (BSA) involved and the overall plaque morphology. The lattice portion of the LS-PGA is performed by computerized algorithm. The LS-PGA shows good correlation with both the PASI and PGA-Ps, and has very good intrarater and interrater reliability (10, 11). The National Psoriasis Foundation (NPF) has developed the NPF Psoriasis Score (NPF-PS) (5, 12, 13), a responder index that includes 6 domains: induration at 2 target sites, current and baseline BSA, PGA, patient's global assessment, and patient's assessment of itch. To improve intrarater and interrater reliability of the induration score, the NPF-PS utilizes a reference card embossed with elevations that increase at 0.25-mm intervals. The NPF-PS strongly correlates with the PASI and PGA, and better reflects patient perception (13). The assessment of total BSA involved by psoriasis is an important standalone measure (5). However, there are significant issues with the interrater reliability of the visual measurement of BSA (14, 15). Nail psoriasis occurs in as many as 50% of patients with psoriasis (16) and in more than 80% of patients with PsA (17). The Nail Psoriasis Severity Index (NAPSI) is used to evaluate the severity of nail psoriasis and was developed to evaluate the response to the treatment of psoriatic nails in clinical trials (18). The modified NAPSI (mNAPSI) (19), developed to enhance the face validity and feasibility of the NAPSI, demonstrated excellent interrater reliability in the original study (19).

The assessment of joints involves counting the number of tender and swollen joints. Sixty-eight joints are assessed for tenderness and 66 joints are assessed for swelling (3, 4). The assessment of tender joints was found to be reliable in previous studies conducted by a group of expert rheumatologists (20, 21). However, the assessment of swollen joints was found to be less reliable (20, 21). Dactylitis, characterized by diffuse swelling of a finger and/or toe, is an important feature of PsA (22, 23). Measures of dactylitis include the Leeds Dactylitis Index (LDI) (24), the assessment tool used in the Infliximab Multinational Psoriatic Arthritis Clinical Trial (25), and a simple count of all dactylitic fingers and/or toes (26). The assessment of dactylitis based on the count of dactylitic fingers and/or toes was shown to be moderately reliable (20). PGA of arthritis activity is one of the core measurements used in clinical trials of inflammatory arthritides, especially rheumatoid arthritis (27). In PsA, the PGA of PsA activity (PGA-PsA) was considered to be an important but not mandatory domain in randomized controlled trials and longitudinal observational cohorts (7). In a recent multicenter study, the PGA-PsA measured using a 0–100-mm visual analog scale was found to be reliable (28).

The measurements outlined above are now being performed by both dermatologists and rheumatologists in clinical trials for psoriasis and PsA. However, it has not been determined whether the assessments of arthritis and dactylitis by dermatologists and the assessments of the skin and nails by rheumatologists are reliable. Therefore, the purpose of the present study was to determine whether the assessment of arthritis, psoriasis, dactylitis, and nails in patients with PsA by rheumatologists and dermatologists is reproducible.

PATIENTS AND METHODS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. ROLE OF THE STUDY SPONSOR
  9. REFERENCES

Patients.

Twenty patients with PsA with varying degrees of psoriasis and peripheral joint involvement typically seen in rheumatology clinics were selected from the University of Toronto PsA clinic at the Toronto Western Hospital, Toronto, Ontario, Canada. The patients consisted of 11 men and 9 women with a mean age of 51 years, psoriasis duration of 19 years, and PsA duration of 11 years. All of the patients met the CASPAR (ClASsification criteria for Psoriatic ARthritis) criteria for the classification of PsA (29). An attempt was made to include patients with a range of disease activity. When selected to participate in the study by the study organizers (DDG, VC), the median (range) of these 20 patients for the disease activity measures were as follows: 7.5 (0–41) for tender joint count (TJC), 1.5 (0–8) for swollen joint count (SJC), 0 (0–3) for dactylitic fingers and/or toes, 4.45 (0.4–24.6) for the PASI, 3.5% (1–70%) for BSA, and 4.5 (0–10) for fingernails with nail involvement.

Assessors.

The assessors consisted of a group of 10 rheumatologists and 9 dermatologists who were members of the Group for Research and Assessment of Psoriasis and Psoriatic Arthritis (GRAPPA). Although 10 dermatologists were invited, only 9 were able to participate. All of the rheumatologists and dermatologists had extensive experience in the assessment of PsA and psoriasis, respectively. However, there were differences among the assessors in their experience with individual measures. Among the rheumatologists, 70% had >10 years of experience in assessing PsA and extensively used joint and dactylitis counts, and the PGA-PsA. Although the remaining 30% had extensive experience with these measures, their experience was <10 years. With regard to the assessment of psoriasis, 50% had extensive experience of >10 years with the PASI, assessment of nail psoriasis, and PGA-Ps, but only minimal experience with the BSA, NPF-PS, LS-PGA, and mNAPSI. Among the dermatologists, 50% had >10 years of extensive experience with assessing psoriasis using the PASI, BSA, PGA-Ps, and the assessment of nail psoriasis, and the rest had <10 years of experience. Fifty percent had <5 years of experience with the LS-PGA, NPF-PS, and mNAPSI. With regard to arthritis assessment, only 1 assessor had extensive experience of >10 years with joint counts, and none had experience in assessing dactylitis.

Design.

The 20 patients and 19 assessors were divided into 2 groups. There were 10 patients in each group. One group included 5 rheumatologists and 5 dermatologists, and the other included 5 rheumatologists and 4 dermatologists. In each group, all of the patients were assessed by the same assessors according to a Latin square design (which was modified slightly because there was an imbalance in the number of subjects and assessors) (30), which facilitates an analysis of components of variation due to patient, observer, and error, while controlling for the order of assessment.

Clinical assessments.

The following clinical measurements were performed by each assessor on each patient, after a 2.5-hour educational session wherein all of the measurements were reviewed. The assessors completed all of the assessments within 30 minutes.

The BSA was visually determined using the rule that the palm of the patient represents 1% of his/her total body surface and the rule of nines (14).

The assessors scored erythema, infiltration, scaling, and area involved for the head, trunk, upper extremity, and lower extremity, and the total PASI score was calculated (8).

The activity of psoriasis was scored on an 11-point (0–10) numerical rating scale, with 0 representing inactive disease and 10 representing the most active disease. This modification of the PGA-Ps was done because the rheumatologists were more comfortable with using this type of scale as opposed to the visual analog scale or a categorical Likert scale.

The assessors first categorized the BSA involved on a scale from 0 to 6 (category 0 = 0%, 1 = 1–3%, 2 = 4–9%, 3 = 10–20%, 4 = 21–29%, 5 = 30–50%, and 6 = 51–100%) (10). They then rated the average plaque quality across all of the involved areas from 0 (none) to 3 (marked) for each of the descriptors thickness, erythema, and scale. Written definitions for each level of severity were provided. The final LS-PGA score was later calculated using the lattice algorithm.

The NPF-PS includes an assessment of patients and physicians (5). In this study, only the reliability of the physician's assessment was studied. Because this was a single time point study, the BSA was assessed as that currently involved by psoriasis, not as a percentage relative to baseline. The first assessor seeing a patient chose 2 target lesions, and all subsequent assessors evaluated the same 2 target lesions for induration using the NPF-PS reference card (none to severe on a 6-point scale [0–5]). The assessors rated the BSA from 0 to 5 (0 = none, 1 = 1–20%, 2 = 21–40%, 3 = 41–60%, 4 = 61–80%, and 5 = 81–100%) and globally assessed the average induration, erythema, and scaling over all of the lesions on a scale from 0 to 5 (where 0 = none and 5 = severe). Written definitions were provided. The final (modified, since patient-reported items were excluded and current BSA was used) NPF-PS score was then calculated.

Individual fingernails of all of the patients were assessed for the following features: onycholysis and oil drop dyschromia (scored from 0 to 3, where 0 = none and 3 = >30% of the nail), pitting (range 0–3, where 0 = 0 pits and 3 = >50 pits), nail plate crumbling (range 0–3, where 0 = none and 3 = >50% of the nail), leukonychia (present/absent), splinter hemorrhage (present/absent), nail bed hyperkeratosis (present/absent), and red spots in the lunula (present/absent). The mNAPSI score was then calculated (19).

A simple count (0–10) of the number of fingernails with nail changes (NN) was performed.

For the peripheral joint assessment, 68 joints (temperomandibular, sternoclavicular, acromioclavicular, glenohumeral, elbows, wrists, metacarpophalangeal, proximal interphalangeal, distal interphalangeal, hips, knees, tibiotalar and midtarsal joints of the ankles, metatarsophalangeal, and proximal interphalangeal joints of the toes) were assessed for tenderness and 66 joints (all of the above except for hips) were assessed for swelling.

The total number of fingers and/or toes with dactylitis was also recorded.

For the PGA-PsA, each assessor was asked to assess the activity of the PsA of the patient on an 11-point numerical rating scale (range 0–10, where 0 = inactive disease and 10 = the most active disease).

Statistical analysis.

Variance components analyses were conducted for each measurement based on analysis of variance models with a random observer, random patient, and both, excluding and including fixed-order effects to account for temporal trends in the responses while assessing the reliability. Estimates of intraclass correlation coefficients (ICCs) and associated 95% confidence intervals (95% CIs) were obtained (31). Sackett et al (32) suggest that values of kappa in the interval 0.80–1.00 represent excellent agreement beyond chance, in 0.60–0.80 represent substantial agreement, in 0.40–0.60 represent moderate agreement, in 0.20–0.40 represent fair agreement, and in 0.0–0.20 represent poor agreement beyond chance. For the purpose of interpreting our results, we adopted this same classification for the ICC. The overall agreement for each measure between all of the assessors was first calculated. Subsequently, the agreement for each measurement among rheumatologists and dermatologists was calculated and compared. All statistical analyses were performed using SAS statistical software (SAS Institute, Cary, NC).

RESULTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. ROLE OF THE STUDY SPONSOR
  9. REFERENCES

Table 1 shows the disease characteristics of the 20 patients with PsA who participated in the study, as well as the median scores for the measures studied. Table 2 shows the results of the ICC values of the assessments performed by all of the assessors. The mNAPSI showed excellent agreement, the PASI, NN, and TJC showed substantial agreement, the BSA, LS-PGA, NPF-PS, and PGA-Ps showed moderate agreement, and the SJC, number of fingers and/or toes with dactylitis, and PGA-PsA showed fair agreement beyond chance. Among dermatologists, there was substantial agreement on the BSA, PASI, LS-PGA, NPF-PS, mNAPSI, NN, PGA-Ps, and TJC, moderate agreement on the PGA-PsA, and fair agreement on the SJC, but no agreement on the number of fingers and/or toes with dactylitis. Among rheumatologists, there was excellent agreement on the mNAPSI and TJC, substantial agreement on the PASI, NN, and number of fingers and/or toes with dactylitis, moderate agreement on the LS-PGA, NPF-PS, PGA-Ps, and SJC, and fair agreement on the BSA and PGA-PsA. When the 2 groups of assessors were compared, the 95% CIs of the ICCs for all measures except dactylitis overlapped, indicating that the only measure with a significant difference in the ICC between dermatologists and rheumatologists was dactylitis.

Table 1. Disease characteristics of the study patients (n = 20)*
CharacteristicsValue
  • *

    Values are the median (interquartile range) unless otherwise indicated. PsA = psoriatic arthritis; BSA = body surface area involved by psoriasis; PASI = Psoriasis Area and Severity Index; LS-PGA = Lattice System Physician's Global Assessment; NPF-PS = National Psoriasis Foundation Psoriasis Score; mNAPSI = modified Nail Psoriasis Severity Index; NN = number of nails with nail changes; PGA-Ps = physician's global assessment of psoriasis activity; PGA-PsA = physician's global assessment of PsA activity.

  • Based on average scores from all dermatologists.

  • Based on average scores from all rheumatologists.

Men, no.11
Age, mean years51
Duration of psoriasis, mean years19
Duration of PsA, mean years11
BSA (range 0–100)1.58 (1.0–2.4)
PASI score (range 0–72)4.16 (3.33–6.61)
LS-PGA score (range 1–8)2.25 (1.5–3.65)
NPF-PS (range 0–20)1.88 (1.36–2.45)
mNAPSI score (range 0–130)0.81 (0.24–1.46)
NN (range 0–10)6.1 (2.44–7.83)
PGA-Ps score (range 0–10)2.23 (1.75–2.65)
Tender joint count (range 0–68)9.7 (2.55–23.2)
Swollen joint count (range 0–66)2 (1.2–4.85)
Number of fingers and/or toes with dactylitis (range 0–20)0.2 (0–0.7)
PGA-PsA score (range 0–10)2.8 (1.4–3.6)
Table 2. Reliability of using ICCs and 95% CIs for each assessment*
MeasureOverallDermatologistsRheumatologists
  • *

    ICCs = intraclass correlation coefficients; 95% CIs = 95% confidence intervals; see Table 1 for additional definitions.

  • Includes the combined ICCs (95% CIs) using data obtained from all assessors.

  • P = 0.0005 between dermatologists and rheumatologists.

Skin assessments   
 BSA0.47 (0.31, 0.68)0.65 (0.47, 0.82)0.30 (0.11, 0.54)
 PASI score0.66 (0.51, 0.82)0.74 (0.58, 0.87)0.70 (0.53, 0.85)
 LS-PGA0.58 (0.41, 0.76)0.73 (0.56, 0.86)0.42 (0.23, 0.65)
 NPF-PS0.58 (0.42, 0.76)0.66 (0.47, 0.82)0.52 (0.32, 0.73)
 mNAPSI0.80 (0.69, 0.90)0.78 (0.63, 0.89)0.85 (0.74, 0.93)
 NN0.73 (0.59, 0.86)0.77 (0.61, 0.89)0.66 (0.47, 0.82)
 PGA-Ps0.54 (0.37, 0.73)0.66 (0.48, 0.82)0.49 (0.29, 0.70)
Arthritis assessments   
 Tender joint count0.78 (0.65, 0.89)0.73 (0.56, 0.86)0.81 (0.68, 0.91)
 Swollen joint count0.24 (0.12, 0.45)0.31 (0.12, 0.57)0.42 (0.23, 0.65)
 Dactylitis0.29 (0.15, 0.51)0.08 (−0.07, 0.32)0.69 (0.52, 0.84)
 PGA-PsA0.39 (0.23, 0.60)0.50 (0.29, 0.72)0.29 (0.11, 0.54)

Table 3 shows information on the sources of variation in the patients seen by rheumatologists and dermatologists, and shows that the majority of the variance was contributed by patients. However, for the 3 measures that proved to be the least reliable among dermatologists (dactylitis, SJC, and PGA-PsA), there was less variation provided by the patients and a proportionally higher variance due to the assessors (Table 3). Similarly, for those measures that proved to be less reliable among rheumatologists (BSA and PGA-PsA), there was less variation provided by the patients and proportionally higher variance provided by the assessors. Order effect was noted for the assessment of dactylitis by the dermatologists and for the assessment of PGA-PsA by the rheumatologists.

Table 3. Variation in patients with PsA*
MeasureEvaluated by dermatologists, % of total variance dueEvaluated by rheumatologists, % of total variance due
PatientAssessorOrderPatientAssessorOrder
  • *

    See Table 1 for abbreviations.

  • Order of examination (whether seen in the first half or the second half of the day).

BSA72.16.10.542.714.80.0
PASI score79.25.40.175.32.20.0
LS-PGA77.97.20.253.019.90.0
NPF-PS72.611.10.060.514.10.1
mNAPSI82.06.80.187.62.70.0
NN82.82.20.672.65.40.1
PGA-Ps73.06.10.057.918.80.1
Tender joint count78.13.20.184.22.40.0
Swollen joint count44.711.42.152.34.31.3
Dactylitis25.712.86.274.63.40.0
PGA-PsA60.39.60.241.627.22.3

DISCUSSION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. ROLE OF THE STUDY SPONSOR
  9. REFERENCES

To our knowledge, this is the first study investigating the reliability of the assessment of the skin, nails, peripheral arthritis, and dactylitis by physicians (dermatologists and rheumatologists) who manage patients and perform clinical trials in psoriasis and PsA. Reliable assessment of disease is important in patient care, clinical trials, and longitudinal observational studies, as well as proper phenotyping of disease for genetic studies. The assessment tools selected for this study were those currently used in randomized controlled trials.

Overall, there was substantial to excellent agreement among dermatologists and rheumatologists on the assessment of nails, PASI, and TJC, the most commonly used measures in clinical trials (3). The design of this study allowed us to efficiently estimate the variability contributed by patients, assessors, and order of assessment. The PsA patients included provided a spectrum of skin and joint disease; the majority of the variance was contributed by patients. The random-effects model used for the variance components analysis involved an assumption of normally distributed random effects and error terms. Some of the measures studied here had somewhat skewed distributions, and some authors have based analyses on log-transformed data to address this. We elected to maintain the measures on their original scale to ensure that the ICC estimates could be easily clinically interpreted. An alternative approach would be to discretize all of the responses and use weighted kappa statistics (33) for these analyses. We did not pursue this approach because of problematic features of the kappa statistic that are being increasingly recognized (34).

Among dermatologists, as expected, there was substantial agreement on all of the measures used in the assessment of the skin and nails. There was also substantial agreement on the number of tender joints. However, there was no agreement on the number of fingers and/or toes with dactylitis. Among rheumatologists, as expected, there was substantial to excellent agreement on the number of tender joints and the number of fingers and/or toes with dactylitis. However, there was only moderate agreement on the number of swollen joints and only fair agreement on the PGA-PsA. There was substantial agreement on nail assessment. For the assessments of the severity of skin psoriasis, only the PASI had substantial agreement, with the LS-PGA, NPF-PS, and PGA-Ps having only moderate agreement.

The PASI has been the gold standard for the assessment of the severity of psoriasis, and is widely accepted as an outcome measure in clinical trials (5, 9). However, the PASI has a number of drawbacks (5), and the LS-PGA (10) and NPF-PS (12) were developed to overcome some of these limitations. PGA provides a subjective overall evaluation of disease severity (9). In the most recent study comparing the PASI, LS-PGA, and PGA, the PASI performed the best, followed by the LS-PGA and PGA in terms of interrater reliability, although the 95% CI overlapped, indicating that there was no significant difference (11). These measures have not been compared with the NPF-PS. Moreover, to our knowledge, no study has compared the assessment of the skin performed by dermatologists with that performed by rheumatologists. We show that in the hands of the dermatologists, the PASI, LS-PGA, NPF-PS, and PGA-Ps perform equally well and show substantial interrater reliability with overlapping 95% CIs. The PASI had the highest ICC. The interrater reliability among rheumatologists, however, was lower than that obtained for dermatologists. The PASI performed the best compared with the LS-PGA, NPF-PS, and PGA-Ps. However, there was no significant difference between the measures, among rheumatologists, and between rheumatologists and dermatologists, since the 95% CI overlapped. Overall, when all of the assessments were considered together, the PASI performed the best, with an ICC of 0.66. Therefore, although the PASI does have limitations, it has the highest interrater reliability and is probably the most reliable method that can be used in clinical trials until more valid measures are developed. The performance of the LS-PGA and NPF-PS was good, especially in the hands of the dermatologists. The addition of the patients' rating to the NPF-PS would not have made a difference because the addition of a constant to all of the physician's assessment scores for NPF-PS would not affect the ICC.

Assessment of the BSA affected by psoriasis is crucial. It influences composite scores such as the PASI, LS-PGA, and NPF-PS, and the PGA-Ps. However, visual assessment of BSA has significant issues with interrater reliability (14, 15). In this study, the overall reliability was only moderate (ICC 0.47), with the ICC among dermatologists being 0.65 and that among rheumatologists being only 0.30. The relatively poor interrater reliability of BSA has undoubtedly affected the reliability of the composite scores. Further training and a more precise definition for visually assessing BSA (35) will help improve the reliability of this measure and those composite measures that incorporate the assessment of BSA.

In this study, tools used in the assessment of nail psoriasis performed well. Overall, the mNAPSI was found to have excellent agreement among all of the assessors and among rheumatologists, and substantial agreement among dermatologists. The mNAPSI was previously shown to have interrater reliability and construct validity (19). Sensitivity to change has to be demonstrated before the mNAPSI is fully validated. The NN also showed substantial interrater reliability overall (ICC 0.73). Although the 95% CI overlapped, the agreement was better among dermatologists (ICC 0.77) compared with rheumatologists (ICC 0.66). This indicates that the reliability of the instruments used in assessing nail disease can be further improved with training on various manifestations of psoriatic nail disease.

The reliability of peripheral joint assessment in PsA has been demonstrated previously through observers in a single center (36), from multiple Canadian centers (20), and in an international study (21). The TJC was more reliable than the SJC. In our study, we have demonstrated substantial overall agreement in the TJC (ICC 0.78), which is similar to what has been shown among rheumatologists in the previous studies, but only fair agreement on the SJC (ICC 0.24). As expected, there was excellent agreement on the TJC among rheumatologists (ICC 0.81). Interestingly, there was also substantial agreement among dermatologists (ICC 0.73), which was not significantly different from that of rheumatologists. However, assessment of the SJC was less reliable, with an overall ICC of only 0.24, an ICC among rheumatologists of only 0.42, and an ICC of 0.31 among dermatologists. Previous studies have also demonstrated poor interrater reliability of the SJC in PsA (20, 21). Major issues with the definition of joint swelling include distinction between swelling due to active inflammation, swelling due to damage, and that due to chronic synovial thickening. There is an urgent need to standardize the definition of a swollen joint. Further training of assessors is required to improve reliability.

Dactylitis is a marker of disease severity (22). Although there are at least 3 instruments used in the assessment of dactylitis (24–26), in this study we chose to assess the reliability of a simple count of the number of fingers and/or toes with dactylitis. The results show that although there was substantial agreement among rheumatologists (ICC 0.69), there was no agreement among dermatologists (ICC 0.08; 95% CI −0.07, 0.32). In a Canadian study by rheumatologists where the assessment of dactylitis was based on the same method that was employed in our study, the ICC was 0.56 (20). In the INSPIRE (INternational SPondyloarthritis Interobserver Reliability Exercise) study (21), the assessment of dactylitis using the LDI (24) showed a better ICC of 0.70. These results indicate that there is agreement among rheumatologists on the assessment of dactylitis. The results show that there is very poor reliability of the assessment of dactylitis by dermatologists, and therefore training on the definition and assessment of dactylitis needs to be performed. It should be noted that the LDI incorporates a check for the definition of dactylitis (a 10% difference in circumference and at least some tenderness on palpation). Therefore, the use of this instrument should increase the reliability for nonrheumatology specialists.

The PGA-PsA, although not considered to be a mandatory domain in randomized controlled trials and longitudinal observational studies in PsA (7), is nevertheless an important measure of disease activity and has been used in the assessment of response using composite response indices such as the American College of Rheumatology criteria for 20% improvement in disease activity (37) and the Psoriatic Arthritis Response Criteria (26). However, in this study, the PGA-PsA had only fair agreement among the rheumatologists (ICC 0.29), better among the dermatologists (ICC 0.51), and a fair overall agreement (ICC 0.39). This study shows that even among PsA experts, there is only fair interrater reliability. Interestingly, the reliability was better among dermatologists. Therefore, there again needs to be further training in this domain, stressing that the PGA-PsA is the assessment of disease activity at the time of examination of the patient and not overall disease severity, which encompasses other factors such as damage and response to treatment.

In summary, our study involving assessors who are involved in the care of patients with PsA has demonstrated substantial to excellent agreement on the TJC, PASI, NN, and mNAPSI. Measures that involved a global assessment performed better in the hands of dermatologists when compared with rheumatologists, probably reflecting differences in the core training in the respective subspecialties. The only measure that showed a statistically significant difference between the dermatologists and rheumatologists was the assessment of dactylitis. SJCs performed poorly even among rheumatologists. Further training in the assessment of BSA will likely improve reliability of the assessment of BSA and other measures that include BSA, such as the PASI, LS-PGA, and NPF-PS. Further education and training on the assessment of dactylitis and SJC is imperative because they are considered to be important markers of disease activity and severity. There is also a need for further training in PGAs for both psoriasis and PsA. It has previously been shown that training and standardization of procedures even among experienced observers can reduce the interobserver variability and reduce the sample size requirements in clinical trials (38, 39). Recognizing these deficiencies, GRAPPA is developing educational measures to better define various assessment tools. There are plans to have dermatologists and rheumatologists participate in further training and assessment exercises in the near future. The assessment of reliability is an important first step in evaluating the utility of clinical measurements. Important next steps in the validation process include assessment of sensitivity to change (40) and feasibility. We then plan to develop practical guidelines regarding the value of these assessments.

AUTHOR CONTRIBUTIONS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. ROLE OF THE STUDY SPONSOR
  9. REFERENCES

All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Dr. Gladman had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study conception and design. Chandran, Cook, Duffin, Helliwell, Kavanaugh, Kreuger, Langley, Mease, Rosen, Wong, Gladman.

Acquisition of data. Chandran, Gottlieb, Cook, Duffin, Garg, Helliwell, Kavanaugh, Kreuger, Langley, McHugh, Mease, Rahman, Rosen, Salvarani, Thaci, Toloza, Wong, Zhou, Gladman.

Analysis and interpretation of data. Chandran, Gottlieb, Cook, Garg, Kavanaugh, Lynde, Mease, Olivieri, Rahman, Rosen, Thaci, Wong, Zhou, Gladman.

ROLE OF THE STUDY SPONSOR

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. ROLE OF THE STUDY SPONSOR
  9. REFERENCES

This study was supported by unrestricted educational grants from Abbott Canada, Amgen, and Wyeth Canada. The companies had no input into the study design, data collection, data analysis, and writing of the manuscript, nor was their approval sought in the content of the submitted manuscript. The publication of this article was not contingent on the approval of any of the companies.

REFERENCES

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. ROLE OF THE STUDY SPONSOR
  9. REFERENCES
  • 1
    Stern RS, Nijsten T, Feldman SR, Margolis DJ, Rolstad T. Psoriasis is common, carries a substantial burden even when not extensive, and is associated with widespread treatment dissatisfaction. J Investig Dermatol Symp Proc 2004; 9: 1369.
  • 2
    Gladman DD, Antoni C, Mease P, Clegg DO, Nash P. Psoriatic arthritis: epidemiology, clinical features, course, and outcome. Ann Rheum Dis 2005; 64 Suppl 2: ii147.
  • 3
    Gladman DD, Helliwell P, Mease PJ, Nash P, Ritchlin C, Taylor W. Assessment of patients with psoriatic arthritis: a review of currently available measures [review]. Arthritis Rheum 2004; 50: 2435.
  • 4
    Mease PJ, Antoni CE, Gladman DD, Taylor WJ. Psoriatic arthritis assessment tools in clinical trials. Ann Rheum Dis 2005; 64 Suppl 2: ii4954.
  • 5
    Feldman SR, Krueger GG. Psoriasis assessment tools in clinical trials. Ann Rheum Dis 2005; 64 Suppl 2: ii6573.
  • 6
    Gladman DD, Mease PJ, Healy P, Helliwell PS, Fitzgerald O, Cauli A, et al. Outcome measures in psoriatic arthritis. J Rheumatol 2007; 34: 115966.
  • 7
    Gladman DD, Mease PJ, Strand V, Healy P, Helliwell PS, Fitzgerald O, et al. Consensus on a core set of domains for psoriatic arthritis. J Rheumatol 2007; 34: 116770.
  • 8
    Fredriksson T, Pettersson U. Severe psoriasis: oral therapy with a new retinoid. Dermatologica 1978; 157: 23844.
  • 9
    Feldman SR. A quantitative definition of severe psoriasis for use in clinical trials. J Dermatolog Treat 2004; 15: 279.
  • 10
    Langley RG, Ellis CN. Evaluating psoriasis with Psoriasis Area and Severity Index, Psoriasis Global Assessment, and Lattice System Physician's Global Assessment. J Am Acad Dermatol 2004; 51: 5639.
  • 11
    Berth-Jones J, Grotzinger K, Rainville C, Pham B, Huang J, Daly S, et al. A study examining inter- and intrarater reliability of three scales for measuring severity of psoriasis: Psoriasis Area and Severity Index, Physician's Global Assessment and Lattice System Physician's Global Assessment. Br J Dermatol 2006; 155: 70713.
  • 12
    Carlin CS, Callis KP, Krueger GG. Efficacy of acitretin and commercial tanning bed therapy for psoriasis. Arch Dermatol 2003; 139: 43642.
  • 13
    Gottlieb AB, Chaudhari U, Baker DG, Perate M, Dooley LT. The National Psoriasis Foundation Psoriasis Score (NPF-PS) system versus the Psoriasis Area Severity Index (PASI) and Physician's Global Assessment (PGA): a comparison. J Drugs Dermatol 2003; 2: 2606.
  • 14
    Ramsay B, Lawrence CM. Measurement of involved surface area in patients with psoriasis. Br J Dermatol 1991; 124: 56570.
  • 15
    Yune YM, Park SY, Oh HS, Kim DJ, Yoo DS, Kim IH, et al. Objective assessment of involved surface area in patients with psoriasis. Skin Res Technol 2003; 9: 33942.
  • 16
    Griffiths CE, Camp RD, Barker JN. Psoriasis. In: BurnsDA, BreathnachSM, CoxNH, GriffithsCE, editors. Rook's textbook of dermatology. 7th ed. Oxford: Blackwell Science; 2004. p. 35.169.
  • 17
    Williamson L, Dalbeth N, Dockerty JL, Gee BC, Weatherall R, Wordsworth BP. Extended report: nail disease in psoriatic arthritis—clinically important, potentially treatable and often overlooked. Rheumatology (Oxford) 2004; 43: 7904.
  • 18
    Rich P, Scher RK. Nail Psoriasis Severity Index: a useful tool for evaluation of nail psoriasis. J Am Acad Dermatol 2003; 49: 20612.
  • 19
    Cassell SE, Bieber JD, Rich P, Tutuncu ZN, Lee SJ, Kalunian KC, et al. The modified Nail Psoriasis Severity Index: validation of an instrument to assess psoriatic nail involvement in patients with psoriatic arthritis. J Rheumatol 2007; 34: 1239.
  • 20
    Gladman DD, Cook RJ, Schentag C, Feletar M, Inman RI, Hitchon C, et al. The clinical assessment of patients with psoriatic arthritis: results of a reliability study of the Spondyloarthritis Research Consortium of Canada. J Rheumatol 2004; 31: 112631.
  • 21
    Gladman DD, Inman RD, Cook RJ, Maksymowych WP, Braun J, Davis JC, et al. International spondyloarthritis interobserver reliability exercise: the INSPIRE study. II. Assessment of peripheral joints, enthesitis, and dactylitis. J Rheumatol 2007; 34: 17405.
  • 22
    Brockbank JE, Stein M, Schentag CT, Gladman DD. Dactylitis in psoriatic arthritis: a marker for disease severity? Ann Rheum Dis 2005; 64: 18890.
  • 23
    Healy PJ, Groves C, Chandramohan M, Helliwell PS. MRI changes in psoriatic dactylitis: extent of pathology, relationship to tenderness and correlation with clinical indices. Rheumatology (Oxford) 2008; 47: 925.
  • 24
    Helliwell PS, Firth J, Ibrahim GH, Melsom RD, Shah I, Turner DE. Development of an assessment tool for dactylitis in patients with psoriatic arthritis. J Rheumatol 2005; 32: 174550.
  • 25
    Antoni CE, Kavanaugh A, Kirkham B, Tutuncu Z, Burmester GR, Schneider U, et al. Sustained benefits of infliximab therapy for dermatologic and articular manifestations of psoriatic arthritis: results from the Infliximab Multinational Psoriatic Arthritis Controlled Trial (IMPACT) [published erratum appears in Arthritis Rheum 2005;52:2951]. Arthritis Rheum 2005; 52: 122736.
  • 26
    Clegg DO, Reda DJ, Mejias E, Cannon GW, Weisman MH, Taylor T, et al. Comparison of sulfasalazine and placebo in the treatment of psoriatic arthritis: a Department of Veterans Affairs Cooperative Study. Arthritis Rheum 1996; 39: 201320.
  • 27
    Felson DT, Anderson JJ, Boers M, Bombardier C, Chernoff M, Fried B, et al. The American College of Rheumatology preliminary core set of disease activity measures for rheumatoid arthritis clinical trials. Arthritis Rheum 1993; 36: 72940.
  • 28
    Cauli A, Gladman D, Mathieu A, Olivieri I, Ujfalussy I, Scarpa R, et al. Patient and physician perception of disease in psoriatic arthritis (PsA): a multicentre GRAPPA and OMERACT study [abstract]. Arthritis Rheum 2007; 56 Suppl: S264.
  • 29
    Taylor W, Gladman D, Helliwell P, Marchesoni A, Mease P, Mielants H, and the CASPAR Study Group. Classification criteria for psoriatic arthritis: development of new criteria from a large international study. Arthritis Rheum 2006; 54: 266573.
  • 30
    Montgomery DC. Design and analysis of experiments. 3rd ed. New York: John Wiley & Sons; 1991.
  • 31
    Burdick RK, Graybill FA. Confidence intervals on variance components. New York: Marcel Dekker; 1992. p. 127.
  • 32
    Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology: a basic science for clinical medicine. 2nd ed. Toronto: Little, Brown; 1991.
  • 33
    Cook RJ. Kappa. In: ArmitageP, ColtonT, editors. The encyclopedia of biostatistics. 1st ed. Hoboken (NJ): Wiley and Sons; 1998. p. 21606.
  • 34
    Cook RJ. Kappa and its dependence on marginal rates. In: ArmitageP, ColtonT, editors. The encyclopedia of biostatistics. 1st ed. Hoboken (NJ): Wiley and Sons; 1998. p. 21668.
  • 35
    Thomas CL, Finlay AY. The ‘handprint’ approximates to 1% of the total body surface area whereas the ‘palm minus the fingers’ does not [letter]. Br J Dermatol 2007; 157: 10801.
  • 36
    Gladman DD, Farewell V, Buskila D, Goodman R, Hamilton L, Langevitz P, et al. Reliability of measurements of active and damaged joints in psoriatic arthritis. J Rheumatol 1990; 17: 624.
  • 37
    Felson DT, Anderson JJ, Boers M, Bombardier C, Furst D, Goldsmith C, et al. American College of Rheumatology preliminary definition of improvement in rheumatoid arthritis. Arthritis Rheum 1995; 38: 72735.
  • 38
    Klinkhoff AV, Bellamy N, Bombardier C, Carette S, Chalmers A, Esdaile JM, et al. An experiment in reducing interobserver variability of the examination for joint tenderness. J Rheumatol 1988; 15: 4924.
  • 39
    Bellamy N, Anastassiades TP, Buchanan WW, Davis P, Lee P, McCain GA, et al. Rheumatoid arthritis antirheumatic drug trials. I. Effects of standardization procedures on observer dependent outcome measures. J Rheumatol 1991; 18: 1893900.
  • 40
    Husted J, Gladman DD, Cook RJ, Farewell VT. Responsiveness of health status instruments to changes in articular status and perceived health in patients with psoriatic arthritis. J Rheumatol 1998; 25: 214655.