Level of agreement between children, parents, and physicians in rating pain intensity in juvenile idiopathic arthritis

Authors

  • Pablo Garcia-Munitis,

    1. Università di Genova, Unità Operativa Pediatria II, Istituto di Ricovero e Cura a Carattere Scientifico G. Gaslini, Genova, Italy
    Current affiliation:
    1. Hospital de Ninos “Superiora Sor Maria Ludovica”, La Plata, Argentina
    Search for more papers by this author
  • Marcia Bandeira,

    1. Università di Genova, Unità Operativa Pediatria II, Istituto di Ricovero e Cura a Carattere Scientifico G. Gaslini, Genova, Italy
    Current affiliation:
    1. Hospital Pequeno Principe, Curitiba, Brazil
    Search for more papers by this author
  • Angela Pistorio,

    1. Servizio di Epidemiologia e Biostatistica, Direzione Scientifica, Istituto di Ricovero e Cura a Carattere Scientifico G. Gaslini, Genova, Italy
    Search for more papers by this author
  • Silvia Magni-Manzoni,

    1. Università di Pavia, Istituto di Ricovero e Cura a Carattere Scientifico Policlinico S. Matteo, Pavia, Italy
    Search for more papers by this author
  • Nicolino Ruperto,

    1. Università di Genova, Unità Operativa Pediatria II, Istituto di Ricovero e Cura a Carattere Scientifico G. Gaslini, Genova, Italy
    Search for more papers by this author
  • Ambra Schivo,

    1. Università di Genova, Unità Operativa Pediatria II, Istituto di Ricovero e Cura a Carattere Scientifico G. Gaslini, Genova, Italy
    Search for more papers by this author
  • Alberto Martini,

    1. Università di Genova, Unità Operativa Pediatria II, Istituto di Ricovero e Cura a Carattere Scientifico G. Gaslini, Genova, Italy
    Search for more papers by this author
  • Angelo Ravelli

    Corresponding author
    1. Università di Genova, Unità Operativa Pediatria II, Istituto di Ricovero e Cura a Carattere Scientifico G. Gaslini, Genova, Italy
    • Pediatria II, Istituto G. Gaslini, Largo G. Gaslini 5, 16147 Genova, Italy
    Search for more papers by this author

Abstract

Objective

To investigate the level of agreement between patients, mothers, fathers, and physicians in rating pain intensity in juvenile idiopathic arthritis (JIA), and to identify factors explaining discrepancies between raters.

Methods

Ninety-four children with JIA and their mothers and fathers were asked to rate independently the intensity of present pain and pain in the previous week on a visual analog scale. The physicians rated pain intensity after physical examination. Agreement between raters was determined using intraclass correlation coefficient and Bland and Altman method. Correlations of explanatory variables with discordance in rating pain intensity were determined by univariate and multivariate analyses. Explanatory variables included sex, age, JIA category, disease duration, results of study ratings, joint inflammation measures, and erythrocyte sedimentation rate.

Results

Agreement in rating present pain was moderate between children and mothers, but was poor between children and fathers and children and physicians. The agreement in rating pain in the previous week was moderate between children and mothers and children and fathers. Mother-father agreement was good. Parents and physicians agreed at a moderate level. In multiple regression analyses, only intensity of present pain was significantly associated with discordance within child-mother, child-father, and child-physician dyads.

Conclusion

Children's ratings of pain were only in moderate agreement with those of their parents and were in poor agreement with those of the physicians, whereas the father and mothers agreed at a good level. The intensity of pain was the strongest determinant of discordance between children and other raters.

INTRODUCTION

Pain is a major problem for many children with juvenile idiopathic arthritis (JIA) and can have profound effects on daily activities and quality of life (1). Recent studies have shown that pain is more prevalent in JIA than previously recognized (2). For these reasons, assessment of the intensity of the child's pain is an essential part of the rheumatologic evaluation. Apart from the findings of clinical examination, physicians usually rely on information obtained from the parents, most often from the mother of the child. However, previous analyses of the level of agreement between children and their parents in rating pain intensity have shown conflicting results (3–6). Furthermore, no study has specifically investigated whether fathers are reliable informants of their children's pain. In addition, little is known about the correlation between the subjective pain experiences of children and the objective clinical assessment of the physician.

In the clinical setting, the parents are usually asked to rate pain intensity using the discomfort scale of the Childhood Health Assessment Questionnaire (CHAQ) (7), which is a 10-cm visual analog scale (VAS) that is located at the bottom of the questionnaire, just above the 10-cm VAS for the parent's global assessment of the child's overall wellbeing. It has been suggested, however, that the close proximity of these scales leads to a reciprocal influence between the 2 ratings, which may go beyond the impact that pain may have on the child's well-being (8).

The objective of the present study was to examine the level of agreement between children, mothers, fathers, and physicians in rating pain intensity in JIA. In addition, we sought information on factors explaining discordance between raters. A further aim was to investigate whether and to what extent the discomfort and global assessments in the CHAQ yield interrelated information.

PATIENTS AND METHODS

Patient selection.

The study group comprised all patients who were seen consecutively in the outpatient clinic between January and June 2004 and who had JIA according to the revised International League for Associations of Rheumatology criteria (9), were ages 5–18 years, and were accompanied by both the mother and the father. Families in which one parent did not live with the child on a daily basis (due to work commitments, legal separation, or divorce) were excluded. Because the assessment of enthesalgia in childhood raises distinctive issues (10), patients with enthesitis-related arthritis were excluded. Verbal informed consent was obtained from both the parents and the children (when applicable).

Measures.

Prior to the study visit (usually 15–60 minutes before), the child, the mother, and the father were asked to rate independently the intensity of the child's present pain and pain in the previous week on a double-anchored, horizontal, 10-cm VAS (where anchor 0 = no pain and anchor 100 = very severe pain) and to complete the Italian version of the CHAQ (11). The 2 pain scales were presented to the raters in random order. The VAS used to assess present pain was placed in a separate form, whereas pain in the previous week was assessed using the discomfort scale included in the CHAQ. To investigate whether and to what extent the proximity of this scale to the VAS for the parent's global assessment of the child's overall well-being creates a reciprocal influence between pain and well-being assessment, each parent pair was randomly assigned to rate pain level using the discomfort VAS of the CHAQ either in its normal format or drawn in a separate form, after having completed a modified version of the CHAQ in which the scale was removed. Both children's pain VAS assessments were administered separately from the CHAQ and were anchored with happy and sad faces. Children and parents were simply instructed to place a mark along the line at a level representing pain intensity. In most of the younger patients (generally those <8 years of age), CHAQ completion was conducted in the form of an interview by a physical therapist (AS). Immediately after conducting the rheumatology physical examination (see below), the attending physician (AR) rated the child's present pain intensity on a 10-cm VAS identical to the one used by the parents. Present pain was also rated independently by a second physician (MB) who observed the attending physician's visit. The attending physician was a pediatric rheumatologist with >15 years of experience. The second physician was a foreign pediatric rheumatologist with >5 years of experience in the field who was just starting a fellowship in the study unit.

Clinical assessment.

The medical charts of each patient were reviewed for the following information: sex, age at disease presentation, JIA category, age at study visit, and disease duration from presentation to the study visit. For each patient, the attending physician performed a routine rheumatologic examination, including an interview regarding history, morning stiffness, fatigue, joint pain, functional limitations, and other issues related to disease activity, and a standard joint assessment (by examining a total of 67 joints). On a special form, the attending physician recorded the number of swollen joints, joints with pain upon movement, joints with tenderness, joints with limited range of motion, and joints with active arthritis (defined as the number of joints with swelling or, if no swelling was present, with limitation of movement with either pain upon movement or tenderness). For each joint, the level of pain on motion and tenderness was graded according to the following scale: 0 = none, 1 = mild (patient complains on joint movement or palpation), 2 = moderate (patient withdraws or changes facial expression on movement or palpation), and 3 = severe (patient responds markedly to movement or palpation), as previously described (8). A joint tenderness and pain on motion score was calculated as the sum of the severity ratings obtained for the scores of tenderness and pain on motion in the whole joints. The attending physician also rated his global assessment of the overall disease activity on a 10-cm VAS (0 = no activity, 10 = maximum activity). Laboratory investigations included the erythrocyte sedimentation rate measurement (Westergren method).

Statistics.

Because the primary objective of the study was to investigate the level of agreement between children, mothers, fathers, and physicians in rating pain intensity, the intraclass correlation coefficient (ICC) (12) was chosen as an indicator of reliability. Establishing the minimally acceptable level of reliability (ρ0) and the specified level of reliability (ρ1) between 2 observers at 0.2 and 0.6, respectively, we calculated that the number of subjects necessary to reach a power of 90% with a Type 1 (alpha) error of 0.05 was 37 (13).

Descriptive statistics were reported in terms of means, medians, SDs, and ranges for the continuous variables and in terms of absolute frequencies and percentages for the categorical variables. For the interpretation of ICC values, the following classification was used: <0.4 = poor agreement, 0.4–0.74 = moderate agreement, ≥0.75 = good agreement (14). Agreement was also evaluated by means of the Bland and Altman method (15).

Univariate analyses were first performed to determine correlations of each possible explanatory variable with the absolute amount of discordance between children and mothers, children and fathers, and children and the attending physician in rating pain intensity. The amount of discordance was calculated by subtracting the score of each child from the score of each of the other raters. Possible explanatory variables assessed were all those included in Tables 1 and 2. Spearman's correlation coefficients were calculated for continuous variables, and Kruskal-Wallis test or Mann-Whitney U test were applied for categorical variables. Stepwise multiple regression analyses were performed by entering continuous variables that yielded a Spearman's correlation coefficient ≥0.5 or categorical variables that proved to be statistically significant on a Kruskal-Wallis test or a Mann-Whitney U test as explanatory variables and the absolute amount of discordance between children and the other raters as the dependent variable.

Table 1. Demographic and clinical features of the 94 study patients*
CharacteristicNo. (%)Mean ± SDMedianRange
  • *

    JIA = juvenile idiopathic arthritis.

  • Assessed in each joint on a 0–3 scale (0 = best, 3 = worst).

Female71 (75.5)   
Male23 (24.5)   
JIA category    
 Systemic16 (17)   
 Polyarticular19 (20.2)   
 Oligoarticular persistent30 (31.9)   
 Oligoarticular extended28 (29.8)   
 Psoriatic1 (1.1)   
Age at disease onset, years 4.3 ± 2.93.30.8–13.4
Disease duration, years 5.7 ± 3.15.30.3–12.6
Age at study visit, years 9.9 ± 3.29.55.1–16.8
Number of active joints 4.1 ± 5.620–30
Number of swollen joints 2.8 ± 4.120–24
Number of joints with limited range of motion 3.7 ± 6.220–43
Number of joints with tenderness 2.8 ± 4.510–26
Number of joints with pain on motion 5.3 ± 7.820–41
Joint tenderness score 3.7 ± 6.620–49
Joint pain on motion score 7.7 ± 12.240–67
Erythrocyte sedimentation rate, mm/hour 24.8 ± 21.7184–109
Table 2. Results of study assessments
AssessmentMean ± SDMedianRange
  • *

    On a 10-cm visual analog scale (0 = best, 10 = worst).

  • On a 0–3 scale (0 = best, 3 = worst).

Present pain*   
 Mother2.3 ± 2.81.10–10
 Father2.2 ± 2.61.30–10
 Child1.5 ± 2.20.40–10
 Attending physician2.7 ± 2.81.50–10
 Second physician3 ± 31.80–10
Pain in the previous week*   
 Mother2.7 ± 2.91.50–10
 Father2.8 ± 2.62.40–9.6
 Child2.5 ± 2.81.70–9.8
Childhood Health Assessment Questionnaire   
 Mother0.5 ± 0.50.20–2.2
 Father0.5 ± 0.60.30–2.4
 Child0.4 ± 0.40.20–1.8
Global assessments*   
 Mother2.8 ± 2.82.00–10
 Father2.8 ± 2.62.40–10
 Child1.9 ± 2.31.00–9.6
 Attending physician4.2 ± 3.64.00–10

The relationship between the discomfort and global assessment scales was measured through linear correlation analysis. Comparison of correlation coefficients between assessments made in the original CHAQ or in separate sheets was made using the test of homogeneity among 2 correlation coefficients. All tests were 2 sided and a P value less than 0.05 was considered statistically significant. The statistical package used was the Statistica (StatSoft, Tulsa, OK).

RESULTS

A total of 94 patients, whose main clinical features are presented in Table 1, were included together with their mothers and fathers. None of the approached families declined to participate in the study. The study cohort was representative of the whole spectrum of severity of patients with JIA who are usually seen in a pediatric rheumatology clinic. The results of the study assessments completed by the children, the mothers, the fathers, and the physicians are reported in Table 2. On average, the children's ratings of pain, disability (CHAQ), and overall well-being were consistently lower (meaning less pain and disability and better well-being) than those given by their mothers and fathers, with the difference being more pronounced for present pain and overall well-being assessment. The mean ratings of mothers and fathers were remarkably similar for all assessments, whereas the physicians tended to report higher (i.e., worse) scores than the other raters, particularly concerning the global assessment.

The level of agreement in pain and functional ability (CHAQ) assessment among raters, measured through the ICC, is shown in Table 3. In rating present pain, there was moderate agreement between children and mothers, but only poor agreement between children and fathers and between children and the attending physician. In rating pain in the previous week, there was moderate agreement between both children and mothers and children and fathers. The agreement between mothers and fathers was good in rating both present pain and pain in the previous week. Parents and physicians agreed at a moderate level in rating present pain. The agreement between the 2 physicians was almost perfect. In rating disability, there was moderate agreement between children and mothers and between children and fathers, and good agreement between mothers and fathers.

Table 3. Level of concordance in pain and functional ability assessment among examiners, measured with the intraclass correlation coefficient*
 Present painPain in the previous weekCHAQ
ChildMotherFatherAttending physicianChildMotherChildMother
  • *

    CHAQ = Childhood Health Assessment Questionnaire.

Mother0.45   0.57 0.44 
Father0.310.73  0.560.770.490.8
Attending physician0.260.510.47     
Second physician0.240.520.450.94    

Figure 1 presents the Bland and Altman plots contrasting rating of present pain by the children with that of the other raters. The Bland and Altman method is an easy way to assess the magnitude of disagreement (both error and bias) between observers, to identify spot outliers, and to see whether there is any trend, for example, an increase in the difference between the 2 observations for high values of the average of the 2 measurements. The bias estimate and 95% limits of agreement were 0.80 and −4.26, 5.85, respectively, for the mother-child dyad; 0.75 and −4.78, 6.28, respectively, for the father-child dyad; and −1.18 and −7.06, 4.7, respectively, for the physician-child dyad. All plots showed good concordance at the lowest levels of pain and progressive increase of discordance with increasing pain intensity; the concordance tended to improve in the few patients who had the highest levels of pain.

Figure 1.

Bland and Altman plots for present pain assessment by children, mothers, fathers, and attending physician.

Univariate analysis of correlation between the absolute amount of discordance within the child-mother, child-father, and child-physician dyads and explanatory variables was performed only for the present pain because there was more disagreement for this type of pain. Categorical variables, which included sex and JIA category, did not yield significant correlations for all 3 dyads. Continuous variables that yielded a Spearman's coefficient ≥0.5 (and thus qualified for inclusion in multivariate analysis) are reported in Table 4. Notably, no correlation was found between children's age and disease duration and the amount of discordance within all 3 dyads.

Table 4. Univariate correlations between the absolute amount of discordance among raters and the explanatory variables*
VariableDiscordance child-motherDiscordance child-fatherDiscordance child-physician
  • *

    Values are the Spearman's correlation coefficient. Only correlations that yielded a Spearman's correlation coefficient ≥0.5 are reported.

Mother's rating of present pain0.730.54
Mother's rating of pain in the previous week0.630.560.54
Mother's global assessment of the child's overall well-being0.580.580.54
Father's rating of pain in the previous week0.550.65
Father's rating of present pain0.520.70
Father's global assessment of the child's overall well-being0.510.54
Child's rating of pain in the previous week0.500.50
Physician's rating of present pain0.68
Physician's global assessment of overall disease activity0.61
Joint tenderness score0.54
Joint pain on motion score0.52

In multiple regression analyses, only intensity of present pain as rated by the mother (P = 0.0002), the father (P = 0.002), and the attending physician (P = 0.009) was significantly associated with the amount of discordance within the child-mother, child-father, and child-physician dyads, respectively. Notably, the children's ratings of pain intensity did not prove to be significantly related to discordance in any model.

To investigate whether the close proximity of the discomfort and global assessment scales in the CHAQ form yields interrelated information, we randomly asked 49 parent pairs to rate the intensity of pain in the previous week using the discomfort scale of the CHAQ in its normal format, and 45 parent pairs to rate pain using a scale drawn in a different sheet. The level of correlation between pain and overall well-being in the former and the latter group was 0.93 and 0.79 (P = 0.005), respectively, for the mothers and 0.89 and 0.73 (P = 0.02), respectively, for the fathers. The significantly greater correlation observed when the 2 scales were placed on the same sheet confirms that the close proximity of the discomfort and global assessment scales in the CHAQ leads to a reciprocal influence of the 2 assessments, which is not completely explained by the impact that pain may have on the child's well-being.

DISCUSSION

In the clinical evaluation of a child with JIA, the observed intensity of pain plays an important role in orienting medical decisions concerning therapy. Because the experience of pain is subjective and personal, children's self reports of pain are given preference in clinical settings when possible (16). However, proxy reporting of pain, generally through the discomfort scale of the CHAQ (7), is standard practice both in the clinical situation and in research studies. It has been suggested that using only parent proxy reports instead of patient self reports would fail to capture the fact that parents and children may differ in their perceptions of health (5). Therefore, there is a growing need to understand the relationship between parent proxy reporting and patient self reporting.

We found that, on average, children tended to provide lower ratings, particularly concerning intensity of present pain and overall well-being, compared with their parents, suggesting either that children may cope with their disease better than realized by parents or that parents tend to be overly solicitous to their children's health problems. ICC analyses confirmed that parent ratings of pain were only in moderate agreement with those of their children. A similar level of concordance was observed for the assessment of physical disability using the CHAQ. This contrasts with the view that parents are more capable of judging disability than pain (5). The finding that the level of agreement between parents and children was only moderate indicates that parents are acceptable informants of their children's health status, but implies that reliance only on parental reports may provide insufficient information on the impact of chronic arthritis on children's well-being. The optimal clinical assessment must, therefore, independently assess pain and disability from the perspective of both the child and the parents and try to integrate the different information in the clinical judgment.

To our knowledge, no study has evaluated specifically whether the fathers are reliable proxy reporters of their children's health status. Because fathers usually spend less time with their children, they are generally regarded as being less appropriate informants compared with mothers. However, because the fathers sometimes attend the clinic with their children and may be the only available parent, it is important to investigate whether their perceptions are similar to those of the mothers. In our analysis, the level of agreement between fathers and mothers was good for all assessments, suggesting that the fathers are as reliable as the mothers as proxy reporters for their children. In the clinical setting, before asking the father to make any assessment, it must be ensured that he lives with his child on a daily basis.

On average, the attending physician provided higher (i.e., worse) ratings for both pain and global assessment compared with children and parents. The agreement between the physician and the children in rating pain was poor, whereas the physician agreed with the parents at a moderate level. The physician's estimation of pain intensity is based on the combination of an interview of the patient and the parents regarding joint pain, morning stiffness, fatigue, functional limitations, and other issues related to disease activity and the results of physical examination, which is particularly focused on the degree of joint tenderness and pain upon passive motion. However, the objective measures of joint inflammation are subjective in reality and may be further biased by observation of the child's behavior towards pain. Although a relationship between increased disease activity and increased pain report in children with JIA has been demonstrated (17, 18), it has been suggested that objective measures of joint inflammation may not serve as an independent variable when trying to determine the relationship between inflammation and subjective pain (19–21). Furthermore, disease-related factors have been shown to account for a relatively small percentage of the variance among factors determining the subjective pain intensity of patients with JIA (16, 22). It is likely that several interrelated factors combine to determine a given child's subjective pain experience, inflammation being only one of these factors. Therefore, the disagreement between the children and the physician is not surprising. The very good agreement between the attending physician and the foreign fellow suggests that the clinical judgment about patients' pain is similar across different health care systems. However, the fact that the former physician demonstrated his examination to the colleague may have influenced her rating, and may partly explain the high consistency.

On multiple regression analyses, only intensity of present pain was consistently associated with parent-child and physician-child disagreement about the child's level of pain. This means that when the parents or the physician perceived greater intensity of pain, there was more discordance with children in rating the level of pain. This phenomenon was also demonstrated by the Bland and Altman plots of pain ratings, which revealed good concordance for the lowest levels of pain and increasing discordance with increasing pain intensity. Notably, children's age and sex were not correlated with the amount of discordance both in univariate and multivariate analyses. Although some studies have suggested that subjective pain report depends in part on the child's cognitive capabilities, with increased reported pain intensities as the child's age increases, several other studies have not shown age to be a statistically significant predictor of pain in children with JIA (1). The impact of sex is also unclear because the results of previous studies have been inconclusive (1).

By randomizing parents to rate the intensity of pain in the previous week using the discomfort scale of the CHAQ or a scale drawn in a different sheet, we found that the level of correlation between pain and overall well-being assessments was significantly higher in the former than in the latter group. This confirms our previous impression that the close proximity of these scales in the CHAQ leads to a reciprocal influence between the 2 assessments, which may go beyond the impact that pain may have on the child's well-being (8). This finding suggests that the discomfort VAS should be administered separately from the CHAQ.

Our study should be viewed in light of certain limitations. We have not investigated the influence of non–disease-related factors, such as child or parent depression, which have been found to correlate with disagreement (6), or psychosocial issues, such as coping strategies, anxiety, family functioning, and family pain history, which are important contributors of pain perception (1, 2). Furthermore, we used different VAS scales for child self report and proxy reports (parents, physicians). The use of VAS scales anchored with happy and sad faces for children only presents several problems because scales with smiling face anchors, compared with neutral face anchors, may alter a child's concept of pain and may confound affective states with pain (23), resulting in inflated pain scores. Several literature reports have shown that children younger than age 7–8 years can not understand the concept of VAS even when provided with a teaching session prior to its use (24, 25). Also, some studies have suggested that VAS scales are poorly reliable for children in general (24–26). Another limitation of our study is that it lacks the interdisciplinary, multidimensional, and comprehensive approach, combining self-report, behavioral, cognitive, socioenvironmental, medical, and biologic parameters that are required for an ideal assessment of pain in children (3). Finally, our patient sample was almost exclusively white; therefore, we cannot determine whether our findings would generalize to a more ethnically diverse population.

In summary, we found that children's pain ratings were only in moderate agreement with those of their parents and in poor agreement with those of the attending physician, whereas the fathers and mothers agreed at an excellent level. The intensity of present pain was the major determinant of discordance between the children and the other raters. These results indicate that parents and children often differ in their perceptions of health, and therefore pain should be independently assessed from the perspective of both the child and the parents. Future research should incorporate measures of psychosocial distress and coping strategies as determinants of discordance in pain perception among children and their parents.

Ancillary