To develop and validate a parent-centered and a child-centered composite disease assessment index for juvenile idiopathic arthritis (JIA): the Juvenile Arthritis Parent Assessment Index (JAPAI) and the Juvenile Arthritis Child Assessment Index (JACAI), respectively.
The JAPAI and the JACAI include 4 measures: parent/child rating of overall well-being, pain, physical function, and health-related quality of life (HRQOL). Validation analyses were conducted on nearly 5,000 patients and included assessment of construct validity, discriminant validity, responsiveness to change, and reliability. Besides the 4-item version, a 3-item version of both indices, which did not include HRQOL, was tested.
The JAPAI and the JACAI demonstrated good construct validity, yielding high correlations with the Juvenile Arthritis Disease Activity Score and moderate correlations with physician global rating and joint counts. Correlations obtained for the JAPAI and the JACAI and for the 4-item and the 3-item versions were comparable. Factorial analysis by principal component analysis showed that both indices are monodimensional. Both the JAPAI and JACAI discriminated well between different disease states and courses and between different levels of American College of Rheumatology Pediatric criteria in a clinical trial, and revealed fair responsiveness to clinical change. Internal consistency was satisfactory, with a Cronbach's alpha of >0.80 in all but 1 of the patient samples tested.
The JAPAI and the JACAI were found to be valid instruments for assessment of disease status in JIA and suitable surrogates of physicians' evaluations. Both indices are potentially applicable in clinical practice, observational studies, and therapeutic trials.
Assessment of disease activity is a central component of the clinical evaluation of children with juvenile idiopathic arthritis (JIA). Because no single measure can serve as a gold standard to assess and monitor disease activity in all of the individual patients, multiple single variables combined in a core set (1) or pooled in a composite score (2) are used. Core set or composite measures are considered more informative than single measures and have the advantage of avoiding the methodologic and statistical problems that arise when multiple measures are employed individually as end points in clinical trials.
Composite scores, which yield a summary number on a continuous scale, are better suited than core set measures for the assessment of the actual level of disease activity in an individual patient. Recently, a composite disease activity score for JIA, the Juvenile Arthritis Disease Activity Score (JADAS), has been created (2). The JADAS includes 4 measures: physician global rating, parent/child global rating, active joint count, and erythrocyte sedimentation rate (ESR). However, although the JADAS incorporates the parent/child global rating, it lacks other important parent-/child-reported outcomes (PCROs), such as pain, physical function, and health-related quality of life (HRQOL). For this reason, it may not sufficiently reflect the parent's or child's perception of the disease burden.
In recent years, increasing attention has been paid to PCROs in JIA (3–6). Incorporation of these measures in patient assessment is deemed important because they reflect the parents' and children's perception of the disease course and the effectiveness of therapeutic interventions. Recently, composite disease activity scores entirely based on patient-reported outcomes have been developed for adult rheumatoid arthritis (7–12). These measures have been found to be of value both in standard clinical care and as trial end points.
At present, such measures do not exist for JIA. For this reason, the purpose of the present study was to develop and validate a parent-centered and a child-centered composite disease activity score for JIA: the Juvenile Arthritis Parent Assessment Index (JAPAI) and the Juvenile Arthritis Child Assessment Index (JACAI), respectively.
Significance and Innovations
This study describes the development and initial validation of 2 new parent-centered and child-centered disease assessment indices for juvenile idiopathic arthritis.
The composite indices integrate information from the most commonly used parent-/child-reported outcomes into a continuous measure.
The composite indices were found to be valid instruments for assessment of disease status in juvenile idiopathic arthritis and suitable surrogates of physicians' evaluations.
Incorporation of the new indices in standard pediatric rheumatology practice and clinical trials may help appraise the parent's and child's perception of the disease status and effectiveness of therapeutic interventions.
PATIENTS AND METHODS
Development of the JAPAI and JACAI.
The JAPAI and JACAI were devised by a panel of 8 pediatric rheumatologists (AC, NR, BL, NS, SP, GCV, AM, AR) with 3 to >20 years of clinical experience in the field, who reached consensus on the individual measures to be included in the indices. Investigators were asked to base their choice on their clinical experience and on a review of the literature on PCROs in JIA (3–6) and patient-centered composite scores in adult rheumatoid arthritis (7–12). After extensive discussion, all investigators agreed that both scores should include the following 4 items: 1) parent/child rating of overall well-being, 2) parent/child rating of pain intensity, 3) assessment of physical function, and 4) assessment of HRQOL. These measures are the most widely used PCROs in clinical practice and research and are those for which more data are available in the literature.
The study protocol was approved by the Ethics Committee of the Istituto G. Gaslini, Genoa, Italy.
Assessment of the JAPAI and JACAI.
Items included in the JAPAI and JACAI were assessed using different scales, depending on the patient data set (see below). The parent/child ratings of well-being and pain were assessed on a 10-cm horizontal-line visual analog scale (VAS) or a 21–numbered circle VAS (where 0 = best and 10 = worst for both VAS) (13). The latter VAS format has been in use at the authors' center after 2007. Physical function was assessed with the Italian parent version of the Childhood Health Assessment Questionnaire (C-HAQ; where 0 = best and 3 = worst) (14) or the Italian parent and child versions of the Juvenile Arthritis Functionality Scale (JAFS; where 0 = best and 30 = worst) (15). HRQOL was assessed with the Italian parent version of the Child Health Questionnaire (CHQ) (14) or the Italian parent and child versions of the Pediatric Rheumatology Quality of Life scale (PRQL; where 0 = best and 30 = worst) (16). For the sake of homogeneity and to ensure that each measure was weighted equally in the composite indices, the 0–3 score of the C-HAQ and the 0–30 score of the JAFS and the PRQL were converted to a 0–10 scale through division by 0.3 and division by 3, respectively. The CHQ comprises 15 subscales and yields 2 summary scores: the physical summary score and the psychosocial summary score. These scores are obtained from the aggregation of 10 of the 15 subscales and have been standardized to have a mean ± SD of 50 ± 10. However, since the 2 summary scores do not have fixed minimum and maximum values, for the purposes of this study we divided the 0–100 score of the 15 CHQ subscales by 10, thus obtaining a 0–10 score for each subscale. The score of the 15 subscales was then averaged to obtain a 0–10 score, which was retained as the “CHQ adapted score.” Because the CHQ score goes in the opposite direction of the scores of the other measures, i.e., higher values indicate better HRQOL, the CHQ score to be included in the JAPAI and JACAI was obtained through the following formula: 10 − CHQ adapted score. This approach to score conversion is similar to that followed in the adjustment of questionnaire scores included in adult patient-centered composite indices (8, 10).
Two different versions of the JAPAI and JACAI were tested in validation analyses. The JAPAI-4 and the JACAI-4 include all 4 items; the JAPAI-3 and the JACAI-3 include only the first 3 items (HRQOL assessment is excluded). After score adjustment, the JAPAI and JACAI scores are obtained as the simple linear sum of the scores of their components, which yields global scores of 0–40 for the 4-item versions and 0–30 for the 3-item versions. The individual elements included in the composite indices examined in the study and their theoretical ranges are shown in Table 1.
Table 1. Composition and theoretical range of the parent-centered and child-centered composite scores tested in the study*
JAPAI = Juvenile Arthritis Parent Assessment Index; JACAI = Juvenile Arthritis Child Assessment Index; VAS = visual analog scale; C-HAQ = Childhood Health Assessment Questionnaire; JAFS = Juvenile Arthritis Functionality Scale; CHQ = Child Health Questionnaire; PRQL = Pediatric Rheumatology Quality of Life scale.
See the main text for the formulae used to normalize the physical function and health-related quality of life questionnaire scores.
The JADAS was computed by assessing the following variables: 1) physician global rating of overall disease activity, measured on a 10-cm horizontal-line VAS or a 21–numbered circle VAS (where 0 = no activity and 10 = maximum activity for both VAS) (16); 2) parent/child rating of well-being (assessed as indicated above); 3) number of joints with active disease, assessed in 27 or 10 joints, as reported (2, 17); and 4) ESR (Westergren method), normalized to a 0–10 scale, as reported (2). The JADAS was calculated as the sum of the scores of its 4 components, which yields a global score of 0–57 and 0–40 for the JADAS-27 and the JADAS-10, respectively. Additional physician-reported measures were the counts of joints with swelling, tenderness/pain on motion, and restricted motion (18). In the clinic patient sample 2 (see below), physicians, parents, and children were asked to rate subjectively and independently the disease status at the time of the baseline visit as remission, continued activity, or flare, and the disease course at the second study visit, as compared from the baseline visit, as much improved, slightly improved, stable, slightly worsened, or much worsened. C-reactive protein (CRP) level was determined with nephelometry.
Study data sets.
Four samples composed of patients meeting the International League of Associations for Rheumatology criteria for JIA (19) were used to validate the JAPAI and the JACAI. The first was a cross-sectional sample of 434 unselected patients who underwent a clinic visit in the authors' units between 2002 and 2007 (clinic patient sample 1). The second sample was also cross-sectional and included 3,324 patients enrolled in study on HRQOL conducted by the Paediatric Rheumatology International Trials Organisation (20). The third sample was composed of 595 patients with polyarthritis included in a controlled trial that compared intermediate versus higher doses of methotrexate (MTX) (21). This sample included a baseline visit and a 6-month visit. The fourth sample was composed of 618 consecutive patients who underwent a clinic visit in the authors' units between 2007 and 2009 (clinic patient sample 2). This sample included a baseline visit and a second visit. In the first 3 samples, subjective ratings were made on 10-cm horizontal-line VAS and physical function and HRQOL were assessed through the C-HAQ and the CHQ, respectively. In the fourth sample, subjective ratings were made on 21–numbered circle VAS and physical function and HRQOL were assessed through the JAFS and the PRQL, respectively. Children's assessments were available only in the fourth data set.
Validation of the JAPAI and JACAI was based on evaluation of construct validity, discriminant validity, responsiveness to change, and reliability.
Construct validity is a form of validation that seeks to examine whether the construct in question, in this case the JAPAI and the JACAI, is related to other measures in a manner consistent with a priori prediction. Based on literature data showing only moderate correlations between PCROs and physician-centered outcomes (22), it was predicted that their correlation with the JADAS and the swollen and tender joint counts would be moderate. Correlations with restricted joint count were predicted to be low to moderate because this measure combines the effect of both disease activity and damage. Correlations with CRP level were predicted to be low because it is known that PCROs correlate poorly with acute-phase reactants. Correlations were assessed on both cross-sectional data and score changes between 2 subsequent visits. All correlations were calculated using Spearman's rank statistics and were considered high, moderate, or poor when >0.7, 0.4–0.7, or <0.4, respectively (23).
To substantiate that the combination of variables included in the JAPAI and JACAI truly measures a single construct, factorial analysis by principal component analysis (24) was carried out on all study data sets.
Discriminant ability was assessed in the clinic patient sample 2 by comparing absolute scores of composite indices in patients judged as being in remission, continued activity, or disease flare by the physician. Furthermore, patients in the same sample were defined, based on the physician's external rating of the disease course at the second visit, as improved (much or slightly), stable, or worsened (much or slightly). Patients in the MTX trial were divided by their American College of Rheumatology Pediatric (ACR Pedi) criteria (1) level of improvement at 6 months into 4 mutually exclusive groups: 1) nonresponders, 2) responders at 30%, 3) responders at 50%, and 4) responders at 70%. Comparisons of absolute scores or score changes among the groups were made by means of the nonparametric analysis of variance (Kruskal-Wallis test); Dunn's test was chosen as an a posteriori test to assess the statistical significance of differences between pairs of patient groups.
Responsiveness to change in composite indices was assessed by computing the standardized response mean (SRM) on the baseline to 6-month changes in the MTX trial. In the clinic patient sample 2, responsiveness was assessed on changes between the baseline visit and the second visit in patients rated as improved, stable, or not improved at the second visit by the physician, the parent, or the child. The SRM was calculated as the mean baseline to end point change in score divided by the SD of the individuals' change in score (25). According to Cohen (26), the threshold levels for SRM were defined as follows: ≥0.20 = small, ≥0.50 = moderate, and ≥0.80 = good.
To investigate test–retest reliability, we calculated the intraclass correlation coefficient (ICC) for the JAPAI values at the first and second visits in patients whose disease course was rated as stable by the physician at the second visit (only visits made within 6 months were assessed). For interpretation of ICC values, the following classification was used: <0.4 = poor agreement, ≥0.4 to <0.75 = fair to good agreement, and ≥0.75 = excellent agreement (27).The internal consistency of the JAPAI and JACAI was assessed with the Cronbach's alpha coefficient (28), using the data obtained from all patient data sets. A value of 0.80 was considered acceptable (29).
All statistical tests were 2-sided; a P value of less than 0.05 was considered as statistically significant. The statistical packages used were the Statistica 8 (StatSoft) and the Stata release 11.
The clinical features of the study samples, including distribution of JIA subtypes, were reported previously (2, 30).
Correlations on cross-sectional data.
The Spearman's correlations between the JAPAI, the JACAI, and the JADAS-27, the JADAS-10, and the other JIA clinical measures in cross-sectional patient samples are shown in Table 2. Overall, correlations for both JAPAI versions were better for the clinic patient sample 2 than for the clinic patient sample 1 and the HRQOL study sample. As predicted, JAPAI and JACAI correlations with joint counts were in the moderate range and correlations with acute-phase reactants were low to moderate. Unexpectedly, JAPAI and JACAI correlations with both JADAS versions were in the high range. Correlations yielded by the JACAI in the clinic patient sample 2 were similar to those provided by the JAPAI, with the exception of a greater correlation of the child-centered composite index with the tender joint count. Overall, correlations yielded by the 3-item versions were similar to those yielded by the 4-item versions for both the JAPAI and the JACAI. In the clinic patient sample 2, most of the correlations with the other JIA outcome measures were greater for the JAPAI-3 and JAPAI-4 than for their individual components (data not shown).
Table 2. Spearman's correlations between parent-centered and child-centered composite indices and the JADAS-27, JADAS-10, and juvenile idiopathic arthritis outcome measures not included in the indices on cross-sectional data*
Changes in the JAPAI and JACAI in relation to changes in other clinical measures.
The Spearman's correlations observed for score changes in the MTX trial and in the clinic patient sample 2 are shown in Table 3. Overall, correlations for both JAPAI versions were similar in the MTX trial and clinic patient samples, except for correlations with joint counts, which were lower in MTX trial patients than in clinic patients. As expected, JAPAI correlations with the physician global rating were in the moderate range, JAPAI correlations with joint counts were in the low to moderate range, and JAPAI correlations with acute-phase reactants were in the low range. As seen in cross-sectional data, JAPAI correlations with both JADAS versions were high or moderate to high. Overall, correlations yielded by the JACAI were comparable to those yielded by the JAPAI, and correlations yielded by the 3-item versions were similar to those yielded by the 4-item versions for both indices.
Table 3. Spearman's correlations between the baseline to end point (MTX trial) or first to second visit (clinic patient sample 2) changes in parent-centered and child-centered composite indices and the JADAS-27, JADAS-10, and juvenile idiopathic arthritis outcome measures not included in the indices*
Factorial analysis by principal component analysis was performed for all composite scores in all patient samples (Table 4). Both the JAPAI and JACAI proved to constitute monodimensional instruments. Item loading indicated that all items contribute significantly to the composite scores. The lowest factor loadings were provided by the item physical function.
Table 4. Results of factorial analysis by principal component analysis on parent-centered and child-centered disease assessment indices*
Clinic patient sample 1 (n = 434)
HRQOL study (n = 3,324)
MTX trial (n = 595)
Clinic patient sample 2 (n = 618)
HRQOL = health-related quality of life; MTX = methotrexate; JAPAI = Juvenile Arthritis Parent Assessment Index; JACAI = Juvenile Arthritis Child Assessment Index; C-HAQ = Childhood Health Assessment Questionnaire; JAFS = Juvenile Arthritis Functionality Scale; CHQ = Child Health Questionnaire; PRQL = Pediatric Rheumatology Quality of Life scale.
Variance explained by the factor, %
Parent/child global assessment
Parent/child pain assessment
Figure 1 shows the values of the JAPAI-4 and the JACAI-4 in relation to the physician's external rating of the disease state at the baseline visit in the clinic patient sample 2 and the changes in the indices in relation to the physician's rating of the disease course at the second visit in the clinic patient sample 2, and to the ACR Pedi response in the MTX trial. Both indices discriminated well between different ratings of disease state and disease course and levels of ACR Pedi response. However, post hoc analyses revealed that the JAPAI-4 did not discriminate between ACR Pedi 30 and ACR Pedi 50 responses, and the JACAI-4 did not discriminate between improved and stable patients. Similar results were obtained with the 3-item version of the indices (data not shown).
Responsiveness to clinical change.
The SRMs for the 2 JAPAI and JACAI versions, the JADAS-27, and the JADAS-10 in patients rated as improved, stable, or worsened at the second study visit by the physicians or the parents are shown in Table 5. Both JAPAI versions were strongly responsive to physician-rated improvement, with SRM values of ≥0.8. The JACAI was less responsive than the JAPAI to both physician-rated and parent-rated improvement, whereas the JACAI was more responsive than the JAPAI to parent-rated worsening. Both the JAPAI and JACAI were superior to the JADAS in capturing parent-rated improvement. Overall, SRMs were greater for all composite indices for worsening than for improvement, with the exception of the greater SRMs of the JAPAI in patients rated as improved by the physician. As expected, SRM values were close to 0 for both the JAPAI and JACAI in patients rated as stable by either the physicians or the parents. The SRM in the MTX trial was 0.83 for both the JAPAI-3 and JAPAI-4.
Table 5. Responsiveness of parent-centered and child-centered composite indices, JADAS-27, and JADAS-10 to change between the first and second visit by the physician's and parent's ratings of the disease course at the second visit in the clinic patient sample 2*
Physician rating (n = 256)
Parent rating (n = 390)
Values are the standardized response mean (95% confidence interval)/no. JADAS = Juvenile Arthritis Disease Activity Score; JAPAI = Juvenile Arthritis Parent Assessment Index; JACAI = Juvenile Arthritis Child Assessment Index.
The test–retest reliability for the JAPAI-3 and JAPAI-4 in patients whose disease course from the initial visit was rated as stable by the physician at the second visit was good, with ICCs of 0.64 and 0.65, respectively. The internal consistency of the JAPAI and JACAI, measured with Cronbach's alpha coefficients, was >0.8 in all patient data sets, except for a value of 0.75 for the JAPAI-3 in the clinic patient sample 1. Removal of each item at a time led to a slight worsening of Cronbach's alpha, with the exception of the physical function item in the 3-item versions of the composite scores, whose removal led to minimal improvement in internal consistency.
This study describes the development and validation of 2 new parent-centered and child-centered composite disease assessment indices for JIA. These indices combine information from global rating of well-being, rating of pain intensity, assessment of physical function, and assessment of HRQOL into a continuous measure. Two different versions of each index were tested: a 4-item version, which includes all 4 measures, and a 3-item version, which does not include HRQOL.
Two of the clinical measures included in the JAPAI and JACAI (global rating of well-being and assessment of physical function) are part of the ACR Pedi core set of outcome measures (1). These variables were identified through a comprehensive statistical and consensus formation process and are universally recognized as the central measures of disease activity in all phenotypes of JIA. The choice of the other 2 variables to be incorporated in the indices (pain rating and HRQOL assessment) was based on their paramount importance among the PCROs used in JIA (3, 4, 31, 32) and was agreed upon by a panel of 9 pediatric rheumatologists with a wide range of clinical experience. Altogether, these processes ensure the face and content validity of the instruments.
The scores of the JAPAI and JACAI result from the arithmetic sum of the scores of each individual component, which makes their calculation simple and quick. The global well-being and pain ratings are both measured on a 0–10-cm VAS. To give equal weight to all measures included in the indices, scores obtained from any tool used to assess physical function and HRQOL are converted to a 0–10 scale with simple formulae. To provide adequate strength to the validation process, construct validity, discriminant validity, responsiveness to clinical change, and reliability of the JAPAI and JACAI were assessed using 4 patient samples consisting of nearly 5,000 patients from several different countries. These patients are likely representative of the entire spectrum of JIA severity.
Both the JAPAI and JACAI demonstrated good construct validity in both cross-sectional and longitudinal samples by yielding strong correlations with the JADAS and fair correlations with JIA outcome measures not included in the indices, such as the physician global rating and the joint counts. This suggests that parent- and child-centered composite indices are suitable indicators of disease status in children with JIA and may serve as surrogates of physician's evaluations. That the JACAI performed similarly to the JAPAI suggests that children are acceptable self-reporters of their disease status. Correlations were comparable across patient samples, which indicates that the indices may be robust enough to cover all disease phenotypes and levels of activity. The results of factorial analysis by principal component analysis showed that both tools were remarkably balanced, with each component contributing equally.
The JAPAI and JACAI proved able to distinguish well between different levels of clinical improvement or diverse disease states, defined by the external rating of the physician. Furthermore, both indices discriminated well between ACR Pedi response categories in the MTX trials, i.e., proportionately greater changes in the indices corresponded with nonresponse and ACR Pedi 30, ACR Pedi 50, and ACR Pedi 70 responses. These statistical performances were comparable to those previously shown by the JADAS in the same trial (2).
Responsiveness of the JAPAI and JACAI to change over time was in line with expectations, with most SRM values in the moderate or good range in patients judged as improved or worsened from the previous visit, and SRM values close to 0 in patients rated as stable. Responsiveness of the JAPAI to change in the MTX trial was good, with an SRM of 0.83 for both versions. Test–retest reliability between 2 subsequent visits in patients rated as stable by the physician at the second visit was good. Evaluation of internal consistency yielded satisfactory results, with all but 1 of the Cronbach's alpha coefficients being >0.8.
Validation analyses showed that the metrologic properties of the 3- and 4-item versions of the JAPAI and JACAI were overall comparable. We favor the use of the 4-item version because we believe it is important to incorporate HRQOL assessment in the indices. However, investigators are free to choose either of the 2 versions, depending on the objective of the study, the data available, or their personal preference.
Our study should be interpreted in light of some potential limitations, the chief of which is the use of different tools to assess well-being, pain, physical function, and HRQOL in diverse patient samples. In the clinic patient sample 2, we used new instruments recently developed by our group, whereas the other patient samples were assessed with more established measures. However, the statistical performances yielded by the indices including new tools were comparable or even superior to those seen with the indices incorporating established measures. An advantage of the new physical function and HRQOL questionnaires is the simplicity of their score calculation, which adds to the feasibility of the indices. Altogether, calculation of the score of each individual item, conversion of their score to the 0–10 scale, and calculation of the global score may require at least 10–15 minutes for the indices that include the older tools, but less than 5 minutes for the indices that include the newer instruments. We recognize that the way we converted the scores of physical function and HRQOL tools might not have generated proportionally equivalent values. Therefore, weightings may not assure a fair representation of each of the questionnaires. The main strengths of our study lie in the large and multinational patient samples and in the inclusion of children's assessments.
In conclusion, we have developed 2 new parent-centered and child-centered composite disease assessment indices for JIA, which are based on the simple arithmetic sum of 3 or 4 clinical measures. The instruments were found to be feasible and to possess both face and content validity; furthermore, they exhibited good construct validity, discriminant validity, responsiveness to clinical change, and reliability in a large patient population. By documenting these key measurement properties, we have shown that the JAPAI and the JACAI are valid instruments for the assessment of disease status in JIA and are, therefore, potentially applicable in standard clinical care, observational studies, and therapeutic trials.
All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Dr. Ravelli had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study conception and design. Consolaro, Ruperto, Martini, Ravelli.