SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. APPENDIX A

Objective

The Cutaneous Assessment Tool (CAT) is a comprehensive, semiquantitative tool for the assessment of skin disease in juvenile dermatomyositis (DM). The goal of this study was to determine whether alternative scoring methods would shorten the CAT without compromising its measurement characteristics.

Methods

A total of 113 children with juvenile DM were assessed at baseline; 94 were assessed again 7–9 months later. Interrater reliability, internal consistency, construct validity, and responsiveness were obtained using the original scoring method and 2 alternative methods: the maximum and binary scoring methods.

Results

Spearman's correlations of the maximum and binary methods with the original were both 0.98 (P < 0.0001) for the CAT activity score, and 0.96 and 0.98, respectively (P < 0.0001), for the CAT damage score. Values obtained for interrater reliability, internal consistency, construct validity, and responsiveness were similar for all 3 scoring methods. Although there was a trend toward the maximum method having higher interrater reliability and the binary method having higher responsiveness, the confidence intervals were overlapping and no statistically significant differences were observed. Correlation coefficients for the 3 scoring methods with other measures of myositis disease activity and damage were very similar.

Conclusion

The maximum and binary methods of scoring the CAT have measurement characteristics similar to the original method. Adoption of one of these abbreviated scoring methods should increase its acceptability to clinicians and researchers.


INTRODUCTION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. APPENDIX A

Muscle disease, expressed as weakness, poor endurance, and impaired function, is the dominant feature of juvenile dermatomyositis (DM). However, cutaneous involvement is also an important manifestation of both disease activity and disease damage, is a significant source of morbidity, and is associated with poorer outcomes (1, 2). For these reasons, skin disease activity and damage are essential components of the overall assessment of children with juvenile DM.

We have previously described the Cutaneous Assessment Tool (CAT), a comprehensive, semiquantitative tool developed by a multidisciplinary group of pediatric rheumatologists and a dermatologist to assess skin disease activity and damage in children with juvenile idiopathic inflammatory myopathy (3). We have shown that the CAT has appropriate reliability, construct validity, and sensitivity to change, and that it is a promising tool for use in clinical and research contexts (3, 4).

In the development of the CAT, the investigators had the explicit goal of including all of the skin lesions that were important in the assessment of juvenile DM. Detailed descriptions were included for each lesion, and each lesion was further subdivided by characteristics (e.g., erythema, scaling, or ulceration) in order to generate different levels of severity of skin disease activity or damage for each one. Individual lesions and their gradations of severity were assigned weights based on expert opinion of the importance of those features in the assessment of skin disease activity and damage in juvenile myositis. The CAT has been criticized for its length and complexity, which has led to the reconsideration of its scoring in order to simplify the tool.

The goal of this study was to evaluate the use of alternative scoring methods for the CAT. Specifically, we were interested in determining whether alternative scoring methods would shorten the CAT and potentially reduce its completion time without compromising its measurement characteristics.

PATIENTS AND METHODS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. APPENDIX A

Patients.

This study involved reanalysis of previously obtained data (3, 4), rather than the study of new patients. There were 113 children with definite or probable juvenile DM (5) enrolled in this study. They were seen by 11 assessors at 10 tertiary-care pediatric institutions at baseline. Ninety-four were reevaluated 7–9 months later. At one center, 17 children were seen by both a pediatric rheumatologist and a dermatologist within 48 hours of each other. Approval from the local institutional review boards was obtained at each center, and consent was obtained from the parents or legal guardians of all participants.

The characteristics of this cohort have previously been described (6–8). In brief, children were enrolled consecutively at participating centers at any point in their disease course. At the baseline visit, the median disease duration was 19 months (25th percentile 8 months, 75th percentile 33 months) and the median global disease activity and damage scores were 2.1 cm (25th percentile 0.6 cm, 75th percentile 4.3 cm) and 1.2 cm (25th percentile 0 cm, 75th percentile 1.5 cm), respectively, on 10-cm visual analog scales (VAS).

All study participants were evaluated as previously described (3, 4). These assessments included history, physical examination, physician global assessments of disease and skin disease activity and damage (separate 10-cm VAS for all global assessments) (6), the Childhood Myositis Assessment Scale (CMAS) (8, 9), the Childhood Health Assessment Questionnaire (C-HAQ) (7), and manual muscle testing (MMT) (7).

The CAT.

In addition to the above, each study participant was evaluated with the CAT. Briefly, the CAT is a 21-item tool with 10 items assessing skin disease activity, 4 items assessing skin disease damage, and 7 items that separately assess both skin disease activity and damage (3). Activity and damage are assessed independently, and individual lesions may have components of both. Each item of the CAT has a specific definition and description of the characteristics to be assessed. There are 2–7 possible response categories for each item representing different degrees of skin disease activity or damage, to which a priori weighted scores are assigned. The scores for individual activity and damage items are summed to give the CAT activity score (potential range 0–96) and the CAT damage score (potential range 0–20), with higher scores corresponding to greater activity and damage.

Alternative scoring of the CAT.

The scoring method described above (original method) was compared with 2 alternative scoring methods. In the first (maximum method) all items were assessed as present or absent. If an item was present, it received the maximum possible score for that item under the original method. The activity and damage items were summed as in the original method to give the maximum CAT activity score (potential range 0–96) and damage score (potential range 0–20). In the second alternative scoring method (binary method) all items were also assessed as being present (scored 1) or absent (scored 0). The activity and damage items were summed to give the binary CAT activity score (potential range 0–17) and damage score (potential range 0–11).

Statistical analysis.

All analyses were performed using the statistical programs SAS, version 8.02 (SAS Institute, Cary, NC) and Intercooled Stata, version 9.2 for Windows (StataCorp, College Station, TX). Descriptive statistics were used, including medians and Spearman's correlation coefficient where appropriate. Measurement characteristics for the original method of scoring the CAT have been reported previously (3, 4).

In order to assess interrater reliability of the 3 scoring methods, the intraclass correlation coefficients (ICCs) for the CAT activity and damage scores for the maximum and binary methods were calculated and compared with the values for the original method (3). Confidence intervals (CIs) were calculated (10).

Internal consistency was assessed with Cronbach's alpha, calculated for the total scores, and with each item deleted serially from the total score. Cronbach's alpha was calculated using standardized variables. The bootstrap function of Stata (StataCorp) was used to calculate 95% CIs with 1,000 iterations to determine each value.

Construct validity of the alternative CAT scoring methods was assessed by calculating Spearman's correlations between the CAT activity scores obtained from the 3 scoring methods and scores from other measures of disease activity (physician global disease activity, physician global skin disease activity, the CMAS, the C-HAQ, and MMT). This was repeated for the CAT damage scores obtained from the 3 scoring methods and scores from other measures of disease damage (physician global disease damage, physician global skin disease damage, the CMAS, the C-HAQ, and MMT).

Responsiveness was assessed by calculating the standardized response mean (SRM) (11) for all children with data at 2 assessments (n = 94). The SRMs for the maximum and binary method CAT activity and damage scores were compared with values for the original method and the 95% CIs were calculated (11).

RESULTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. APPENDIX A

Baseline CAT activity and damage scores based on the original, maximum, and binary methods are presented in Table 1. The Spearman's correlations of the maximum method and binary method CAT activity scores with the original CAT activity score were each 0.98 (P < 0.0001). The Spearman's correlations of the maximum method and binary method CAT damage scores with the original CAT damage score were 0.96 and 0.98, respectively (P < 0.0001).

Table 1. Summary of baseline scores, interrater reliability, internal consistency, and responsiveness for the original, maximum, and binary methods of scoring the Cutaneous Assessment Tool activity and damage scores*
 OriginalMaximumBinary
  • *

    ICC = intraclass correlation coefficient; 95% CI = 95% confidence interval; SRM = standardized response mean.

  • Calculated with serial item deletion. These are minimum and maximum values of Cronbach's alpha that were obtained when each item was serially deleted from the calculation of the total score.

Baseline activity score   
 Median7144
 25th, 75th percentile3, 116, 202, 5
 Minimum, maximum0, 440, 690, 12
Baseline damage score   
 Median111
 25th, 75th percentile0, 10, 20, 1
 Minimum, maximum0, 130, 150, 8
Activity score interrater reliability   
 ICC0.700.620.60
 95% CI0.36, 0.860.28, 0.820.06, 0.83
Damage score interrater reliability   
 ICC0.800.830.65
 95% CI0.52, 0.920.65, 0.920.17, 0.85
Activity score internal consistency   
 Cronbach's alpha0.790.760.76
 95% CI0.74, 0.850.69, 0.820.69, 0.82
 Minimum, maximum0.76, 0.810.73, 0.770.73, 0.77
Damage score internal consistency   
 Cronbach's alpha0.740.700.70
 95% CI0.42, 1.00.41, 0.990.41, 0.99
 Minimum, maximum0.67, 0.760.61, 0.730.61, 0.73
Activity score responsiveness   
 SRM0.520.490.67
 95% CI0.32, 0.720.29, 0.690.47, 0.87
Damage score responsiveness   
 SRM0.020.020.16
 95% CI−0.18, 0.22−0.18, 0.22−0.04, 0.36

Interrater reliability.

As shown in Table 1, the ICCs ranged from 0.60–0.70 for the original, maximum, and binary CAT activity scores, and from 0.65–0.80 for the original, maximum, and binary CAT damage scores. The 95% CIs were overlapping.

Internal consistency.

Cronbach's alpha values for original, maximum, and binary CAT activity scores ranged from 0.76–0.79. Cronbach's alpha values for the original, maximum, and binary CAT damage scores ranged from 0.70–0.74 (Table 1). The 95% CIs were overlapping.

Construct validity.

Spearman's correlations used to compare construct validity of the 3 scoring methods are shown in Table 2. Correlations of the CAT activity scores were high with global disease and skin disease activity, and moderate with measures of muscle strength and function. Correlations of the CAT damage scores were moderate with global disease, skin disease damage, and muscle strength, and lower with measures of physical function. The 3 CAT scoring methods had similar construct validity.

Table 2. Summary of the comparison of the construct validity of the 3 methods of scoring the Cutaneous Assessment Tool (CAT)*
AssessmentNo.Method
OriginalMaximumBinary
  • *

    Construct validity was assessed using Spearman's rank correlations of the CAT activity and damage scores with other measures of disease activity and damage. P < 0.0001 unless indicated otherwise. CMAS = Childhood Myositis Assessment Scale; C-HAQ = Childhood Health Assessment Questionnaire; MMT = manual muscle testing.

  • P ≥ 0.0001 and < 0.05.

  • P ≥ 0.05.

CAT activity score    
 Physician global assessment of skin disease activity960.830.810.81
 Physician global assessment of disease activity980.770.750.76
 CMAS98−0.48−0.46−0.47
 C-HAQ960.400.400.42
 Total MMT score50−0.36−0.39−0.40
CAT damage score    
 Physician global assessment of skin disease damage950.530.540.49
 Physician global assessment of disease damage1130.520.510.47
 CMAS98−0.19−0.18−0.18
 C-HAQ960.130.150.11
 Total MMT score50−0.38−0.34−0.37

Responsiveness.

The SRM for the original, maximum, and binary CAT activity scores demonstrated moderate responsiveness (Table 1). The SRM values for the original, maximum, and binary CAT damage scores were low, as expected, due the relatively short time course of the study.

DISCUSSION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. APPENDIX A

In this article, we have compared 3 methods of scoring the CAT and have shown that the alternative scoring methods have measurement characteristics similar to those of the original scoring method, including interrater reliability, internal consistency, construct validity, and responsiveness.

For interrater reliability, both alternative scoring methods appeared to have somewhat lower ICCs for the CAT activity score, and there was a lower ICC for the binary CAT damage score. However, the 95% CIs for the ICCs of all 3 methods were widely overlapping, and no significant difference in interrater reliability was observed. Internal consistency, as measured by Cronbach's alpha, showed a trend towards the alternative scoring methods having slightly lower values. Again, the 95% CIs were widely overlapping, and no significant difference was observed. The correlations of the CAT activity and damage scores with other measures of disease activity or damage were essentially indistinguishable for the different scoring methods. Finally, the SRMs calculated to assess responsiveness were similar for each of the scoring methods. There was a trend for the SRMs of the binary method to have somewhat higher values, but with the widely overlapping 95% CIs, no significant difference was observed.

Taken together, these results fail to document superiority of any of these scoring methods. This is not surprising, given that it has been shown that changes in weighting of items typically do not affect the measurement characteristics of tools like the CAT (12). Therefore, we conclude that each of these scoring methods is likely to perform similarly. However, both alternative scoring methods simplify the CAT, and it is reasonable to adopt one of these. There is not a clear choice as to which alternative to choose. The binary scoring method showed a trend towards better responsiveness, while the maximum method showed a trend towards better interrater reliability. The binary method results in the simplest CAT because the assessor does not need to remember what score a particular lesion receives; however, the maximum method maintains the weighting of the original method, assigning higher scores for more important lesions, such as skin ulcers. The revised tool and alternative scoring methods can be found online at http://www.niehs.nih.gov/research/resources/collab/imacs/othertools.cfm.

Despite their similarities in performance in this study, the alternative scoring methods have some practical advantages over the original scoring method. The CAT has been criticized for being overly long and complex. We have argued that this allows the CAT to be more comprehensive and therefore to capture the full range of cutaneous disease in juvenile myositis (3). Given that our current results show that the alternative scoring methods perform similarly to the original method, maintenance of this complexity may be unnecessary. Both of the alternative scoring methods convert the CAT to a simple 21-item scale and eliminate the gradations of severity originally described for each lesion. In doing so, the overall tool is shortened and simplified, and can now be represented on a single page. This should increase the acceptability of the CAT, likely decreasing administration time and allowing it to be used more widely in clinical and research contexts.

It is possible that the 3 methods of scoring the CAT would have performed differently if they had been used to assess individual patients with juvenile DM. For example, it is possible that a child with very mild features of a particular lesion would be more likely to be scored as absent using the maximum method than with the binary method because an assessor might be reluctant to assign the highest score to a mild rash. We cannot determine whether this would be the case given that this study only involved rescoring data that had previously been obtained, but factors such as this may lead the measures to perform less similarly than we observed.

In conclusion, we have compared 3 methods of scoring the CAT and shown that these methods perform similarly. Given that the alternative scoring methods allow the CAT to be shortened and thus to become more practical and likely to be accepted by clinicians and researchers, we recommend that an abbreviated form of the CAT be adopted. Future work will be needed to ensure that the CAT continues to perform well in its new format.

AUTHOR CONTRIBUTIONS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. APPENDIX A

Dr. Huber had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study design. Dugan, Miller, Rider.

Acquisition of data. Dugan, Rider.

Analysis and interpretation of data. Huber, Lachenbruch, Dugan, Miller, Rider.

Manuscript preparation. Huber, Dugan, Miller, Rider.

Statistical analysis. Huber, Lachenbruch.

Acknowledgements

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. APPENDIX A

We thank Drs. Edward Cowen and Michael Ward for critical reading of the manuscript.

REFERENCES

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. APPENDIX A
  • 1
    Hundley JL, Carroll CL, Lang W, Snively B, Yosipovitch G, Feldman SR, et al. Cutaneous symptoms of dermatomyositis significantly impact patients' quality of life. J Am Acad Dermatol 2006; 54: 21720.
  • 2
    Huber AM, Lang B, LeBlanc CM, Birdi N, Bolaria RK, Malleson P, et al. Medium- and long-term functional outcomes in a multicenter cohort of children with juvenile dermatomyositis. Arthritis Rheum 2000; 43: 5419.
  • 3
    Huber AM, Dugan EM, Lachenbruch PA, Feldman BM, Perez MD, Zemel LS, et al. The Cutaneous Assessment Tool (CAT): development and reliability in juvenile idiopathic inflammatory myopathy. Rheumatology (Oxford) 2007; 46: 160611.
  • 4
    Huber AM, Dugan EM, Lachenbruch PA, Feldman BM, Perez MD, Zemel LS, et al, in cooperation with The Juvenile Dermatomyositis Disease Activity Collaborative Study Group. Preliminary validation and clinical meaning of the Cutaneous Assessment Tool in juvenile dermatomyositis. Arthritis Rheum 2008; 59: 21421.
  • 5
    Bohan A, Peter JB. Polymyositis and dermatomyositis (first of two parts). New Engl J Med 1975; 292: 3447.
  • 6
    Rider LG, Feldman BM, Perez MD, Rennebohm RM, Lindsley CB, Zemel LS, et al, in cooperation with the Juvenile Dermatomyositis Disease Activity Collaborative Study Group. Development of validated disease activity and damage indices for the juvenile idiopathic inflammatory myopathies. I. Physician, parent and patient global assessments. Arthritis Rheum 1997; 40: 197683.
  • 7
    Huber AM, Hicks JE, Lachenbruch PA, Perez MD, Zemel LS, Rennebohm RM, et al, and the Juvenile Dermatomyositis Disease Activity Collaborative Study Group. Validation of the Childhood Health Assessment Questionnaire in the juvenile idiopathic inflammatory myopathies. J Rheumatol 2001; 28: 110611.
  • 8
    Huber AM, Feldman BM, Rennebohm RM, Hicks JE, Lindsley CB, Perez MD, et al, for the Juvenile Dermatomyositis Disease Activity Collaborative Study Group. Validation and clinical significance of the Childhood Myositis Assessment Scale for assessment of muscle function in the juvenile idiopathic inflammatory myopathies. Arthritis Rheum 2004; 50: 1595603.
  • 9
    Lovell DJ, Lindsley CB, Rennebohm RM, Ballinger SH, Bowyer SL, Giannini EH, et al, in cooperation with the Juvenile Dermatomyositis Disease Activity Collaborative Study Group. Development of validated disease activity and damage indices for the juvenile idiopathic inflammatory myopathies. II. The Childhood Myositis Assessment Scale (CMAS): a quantitative tool for the evaluation of muscle function. Arthritis Rheum 1999; 42: 22139.
  • 10
    Fleiss JL, Shrout PE. Approximate interval estimation for certain intraclass correlation coefficient. Psychometrika 1978; 43: 25962.
  • 11
    Beaton DE, Hogg-Johnson S, Bombardier C. Evaluating changes in health status: reliability and responsiveness of five generic health status measures in workers with musculoskeletal disorders. J Clin Epidemiol 1997; 50: 7993.
  • 12
    Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 3rd ed. New York: Oxford University Press; 2003.

APPENDIX A

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. APPENDIX A

MEMBERS OF THE JUVENILE DERMATOMYOSITIS DISEASE ACTIVITY COLLABORATIVE STUDY GROUP

In addition to the authors, the following are members of the Juvenile Dermatomyositis Disease Activity Collaborative Study Group who contributed to this work: Susan H. Ballinger, Suzanne L. Bowyer, Brian M. Feldman, Carol B. Lindsley, Murray H. Passo, Maria D. Perez, Ann M. Reed, Robert M. Rennebohm, Carol A. Wallace, and Lawrence S. Zemel.