* Correspondence to second author at CanChild Centre for Childhood Disability Research, IAHS, Room 408, 1400 Main Street West, Hamilton, Ontario, Canada, L8S 1C7. E-mail: email@example.com
This study assessed stability of measurement of quality of life (QOL) and health-related quality of life (HRQOL) over the course of 1 year among 185 adolescents (mean age 16y, SD 1y 9mo) with cerebral palsy (CP). Participants were classified on the Gross Motor Function Classification System as level I (n=55), II (n=30), III (n=27), IV (n=46), or V (n=27). QOL was assessed by self- (n=125) or proxy-report (n=60) with the Short Version of the Quality of Life Instrument for People with Developmental Disabilities (QOL Instrument), which describes domains of Being, Belonging, and Becoming. HRQOL was captured through parent proxy-reports with the Health Utilities Index Mark 3 (HUI3). Generalizability coefficients (G) for domain and Overall QOL scores on the QOL Instrument ranged from 0.50 to 0.73, indicating that between 50 and 73% of the variance was stable over 1 year. Stability on the HUI3 was excellent (G>0.90) for ambulation and overall utility scores; moderate (G=0.70–0.90) for speech, vision, dexterity, cognition, and hearing; and low for pain (G=0.48) and emotion (G=0.24). Correlations between scores on the two instruments were moderate even when adjustments were made for the lack of perfect stability over 1 year. This supports the notion that QOL and HRQOL are different aspects of life experience among adolescents with CP.
Jack is a 13-year-old adolescent with cerebral palsy (CP). He has a severe visual impairment, is unable to walk or communicate verbally, and requires complete support for basic self-care and activities of daily living. Despite these limitations, Jack participates in a special education program during the week, attends church on the weekend, and is able to live at home with his family on a full-time basis. His mother recognizes that ‘on the surface’ Jack has very severe disabilities, but insists that ‘overall’ he’s at least somewhat satisfied with life. She is generally content with Jack’s current circumstances, but worries what will happen in the future as she ages and Jack continues to grow.
What is the relationship between the ‘on the surface’ and ‘overall’ perspectives expressed by Jack’s mother? To what degree are these perceptions stable over time?
Discussions of quality of life (QOL) often give rise to two perspectives regarding its assessment: one that focuses on markers of biomedical status and activities of daily living (i.e. characteristics that can be observed ‘on the surface’) and another that focuses on holistic (i.e. overall) well-being. Some authors have referred to these perspectives as objective versus subjective;1 objective functioning versus subjective well-being;2 health status versus QOL;3 or health-related quality of life (HRQOL) versus QOL.4
There is clearly a divergence in the literature regarding perspectives on QOL and how they should be conceptualized and measured. The authors have discussed these issues elsewhere and summarized the findings of 20 studies of QOL and HRQOL among children, adolescents, and adults with CP, focusing special attention on adolescence.5 Consistent with previous studies,6 they noted that the assessment of QOL poses significant methodological challenges in a heterogeneous population, such as persons with CP, and observed that emphasis has traditionally been placed on the ‘on the surface’ rather than the ‘overall’ perspective.
Discordance between these two views of QOL has been reported in a cross-sectional study of 203 adolescents with CP.4 The findings showed (1) that correlations between measures of QOL and HROQL were moderate at best, explaining a maximum of 14% of the variance between the two approaches, and (2) that gross motor function (i.e. severity of CP) was significantly associated with a measure of HRQOL (r=−0.81) but not with a measure of QOL. The assessments of QOL and HRQOL clearly captured different aspects of the lives of adolescents with CP, which indicates that these perspectives should be treated as separate dimensions.
In addition to these cross-sectional issues, it is important to determine whether and how much QOL and HRQOL vary over time. We expected that these constructs would be stable among adolescents with CP, at least over the course of 1 year. This expectation was based on the fact that CP is a chronic rather than a degenerative condition, so there was no biomedical reason to assume that ratings of QOL or HRQOL should change systematically over this period. Our expectation was also informed by a study of 177 children and adolescents with CP that revealed no significant difference in HRQOL over 1 year among 9 of the 10 domains on the Child Health Questionnaire.7 For these reasons, the authors expected that QOL and HRQOL would be stable over 1 year among adolescents with CP.
This study involved 185 adolescents with CP (99 males and 86 females) from across the province of Ontario, Canada. Adolescents ranged from 13 to 20 years of age (mean 16y [SD 1y 9mo]) at the time of the first QOL assessment. Participants were classified with the Gross Motor Function Classification System (GMFCS) as level I (n=55), II (n=30), III (n=27), IV (n=46), or V (n=27). Adolescents who self-reported in the QOL interview did not differ significantly from those who proxy-reported in terms of age (t=1.32, p=0.19), sex (χ2=1.50, p=0.22), or urban or rural environment (χ2=0.44, p=0.51). However, adolescents who required a proxy assessment had more severe impairments of gross motor function (χ2=47.0, p<0.001).
Data were collected in the course of the Adolescent Study of Quality of Life, Mobility and Exercise (ASQME) coordinated at the CanChild Centre for Childhood Disability Research in Hamilton, Ontario, Canada. The ASQME is a 5-year continuation of the Ontario Motor Growth (OMG) study, which is reported in detail elsewhere.8 Children were originally included in the OMG study if they had a clinical diagnosis of CP or were strongly suspected to have CP by their treating therapist. Recruitment of participants for the ASQME has been described in a separate report.4 Of the 203 individuals who completed an initial QOL interview, 185 (91.1%) were reassessed approximately 1 year later.
The ASQME was approved by the Research Ethics Board at McMaster University. Written and informed consent was obtained from the caregivers of all adolescents who participated, as well as written assent from the adolescents whenever possible.
Quality of Life Instrument for People with Developmental Disabilities
QOL was assessed with the Short Version of the Quality of Life Instrument for People with Developmental Disabilities (QOL Instrument).9–11 This measure conceptualizes QOL as ‘The degree to which a person enjoys the important possibilities of his or her life,’ or simply, ‘How good is your life for you?’ Nine areas of life are grouped into three domains of QOL: Being – the basic aspects of ‘who one is’; Belonging – the person’s fit with his or her environment; and Becoming – the purposeful activities carried out to achieve personal goals, hopes, and wishes.12
The Short Version of the QOL Instrument consists of 27 items that consider both the importance and satisfaction of various aspects of life (with the exception of items for the subdomains of Psychological and Spiritual Being, which consider only satisfaction). Response options for ratings of importance and satisfaction range from 1 (not important or satisfied) to 5 (extremely important or satisfied). Ordinal item scores are combined by using a multiplicative algorithm to generate continuous QOL scores ranging from –10.0 (not satisfied with extremely important life issues) to 10.0 (extremely satisfied with extremely important life issues). Item responses are then averaged to generate scores for the nine subdomains and three domains, as well as an overall QOL score. The psychometric properties of the QOL Instrument have been evaluated in the current population of adolescents with CP (Livingston et al., unpublished data) and are as good as those reported in a study of 500 adults with developmental disabilities from across Ontario.11
Health Utilities Index Mark 3
HRQOL was measured by proxy using the Health Utilities Index Mark 3 (HUI3), which follows the definition of HRQOL developed by Patrick and Erickson12 as ‘the value assigned to the duration of life as modified by the impairments, functional state, perceptions, and social opportunities that are influenced by disease, injury, treatment, or policy.’13 The HUI3 describes health status in eight attributes, each with five or six ordinal levels.14 Attribute levels cover the full range of possible abilities and disabilities and are meant to be clearly distinguishable from one another.
An algorithm generates a utility score for each attribute and an overall utility score ranging from 1.00 (perfect health) to 0.00 (death). Scores less than zero are also possible and are thought to indicate health states considered worse than death. The utility function is derived from the preferences for health status that members of the general public in Hamilton, Ontario, Canada, placed on each attribute during the development of the HUI3. Utility scores are believed to estimate HRQOL.15
Gross Motor Function Classification System
The GMFCS describes the motor performance of children with CP on the basis of their functional abilities and their need for assistive technology and wheeled mobility.16 Functional levels range from I (independent gross motor function with few limitations) to V (complete dependence for all motor activities). Because adolescent age-band descriptors were not available at the beginning of the current study, classification was based on criteria validated for children aged 6 to 12 years.
A structured interview was used to complete the QOL Instrument. Adolescents were interviewed by physical and occupational therapists, who received training before data collection. Adolescents who were able to communicate were interviewed with the self-report version of the measure, and information from those who could not (as judged by the interviewer) was collected with the proxy version, using a parent or caregiver as the respondent. Caregivers completed the HUI3 as part of the second annual ASQME assessment (completed a mean of 2mo [SD 4mo] before the QOL interview).
Stability over 1 year was assessed with Generalizability Theory.17–19 This approach differs from Classical Test Theory in that it describes the assessment of reliability in terms of facets, which can be thought of as different elements (e.g. participants, items, and time) of the universe being measured. This allows researchers to consider multiple variables at the same time, unlike Classical Test Theory, in which they must be considered separately; i.e., internal consistency to test the effect of the number of items, test–retest reliability to test the effect of time, and interrater reliability to assess the effect of using different observers. In the current study, stability was defined as the degree to which scores obtained during an initial assessment could be generalized to those obtained some time later.
Generalizability coefficients (G) were calculated for self- and proxy-reports on the QOL Instrument such that variance between participant scores (i.e. the variability of scores) was divided by the sum of variance between participant scores, the variance due to the interaction between participant scores and time (i.e. differences between individuals over time), and the variance explained by the main effect of time (i.e. group change over time). This statistic is equivalent to an intraclass correlation coefficient (ICC), representing a single (rather than average) measure of absolute agreement (rather than consistency). Stability of QOL Instrument scores between all participants was estimated by nesting participants within rater type (i.e. self or proxy). Details of the calculation of these coefficients are described elsewhere (Livingston et al., unpublished data).
Stability of HUI3 scores was calculated in a way that is functionally equivalent to the assessment of stability on the QOL Instrument. However, whereas the domain and Overall QOL scores on the QOL Instrument contained multiple items, the single-attribute and overall utility scores on the HUI3 were represented by single values. Thus, items were not included as a fixed facet in the analysis of HUI3 utility scores.
All data were analyzed in SPSS 15.0 for Windows. G coefficients were calculated in Microsoft Excel by extracting mean square values from repeated-measures analysis of variance computed in SPSS. Missing data were imputed for the QOL Instrument on a case-by-case basis, as long as at least one item was completed for each subdomain. Importance ratings that contained missing data were imputed on the basis of importance ratings for other items within that subdomain that were complete. The same strategy was used for ratings of satisfaction. Missing data were not imputed if all ratings for importance or satisfaction were missing in a particular subdomain. No such adjustment was made for the HUI3, because utility scores for each of the eight attributes were represented by single items.
Factors affecting stability of measures over time
Factors affecting stability include the following: (1) the variability of scores between participants; (2) differences between individuals over time (consisting of measurement error and ‘true’ differences between individuals); and (3) group change over time (often measured by comparing the means at each time point and calculating an effect size).20 To have perfect stability, the variability of scores between participants should be high and there should be absolutely no change over time (i.e. factors 2 and 3 should be equal to zero). This scenario is depicted in Figure 1a. Scores for individual participants are represented with dots and are connected between time points by lines.
Imperfect stability is associated with one or more of the following three issues: significant group change over time (Fig. 1b); significant differences between individuals over time (Fig. 1c); and limited variability of scores between participants (Fig. 1d). Note that in Figure 1c there is absolutely no difference between the group means obtained at each time point (i.e. Cohen’s effect size is equal to 0). Stability is less than perfect in this case because individual participants are changing at different rates and in different directions. In Figure 1d, the magnitude of even the slightest variation over time is relatively large compared with the variability of scores, resulting in imperfect stability. Thus, all three factors are important to consider when assessing issues related to stability.
Quality of Life Instrument for People with Developmental Disabilities
The 185 participants who completed two administrations of the QOL Instrument were reassessed a mean of 1 year 1 month (SD 2mo) after the first interview. Missing data were rarely observed, with a frequency of 0.51% for the first assessment and 1.17% for second. No difference was detected in the rate of attrition between participants who self-reported versus those who required a proxy assessment (χ2=0.05, p=0.81).
G coefficients ranged from 0.50 to 0.73 for domain and Overall QOL scores (Table I). This indicated that 50 to 73% of the variance between QOL Instrument scores was stable over 1 year. Inspection of the three terms that comprised each coefficient demonstrated that the lack of perfect stability was not due to the main effect of time but was instead due to the interaction between participant scores and time. In other words, although there was virtually no group change over 1 year, there were moderate differences between individual scores over time.
Table I. Stability of scores on the Quality of Life Instrument for People with Developmental Disabilities over 1 year
Varianceexplainedby the maineffect of time(not a pvalue)c
aVariability of scores. bDifferences between individuals over time. cGroup change over time. dGeneralizability coefficient (single measure of absolute agreement over 1y). QOL, quality of life.
Health Utilities Index Mark 3
Overall utility scores were available for 192 of the 203 adolescents who completed the first QOL interview. Seven participants (3.4%) skipped their second annual ASQME assessment, which meant that their parents did not complete the HUI3, and four others (2.0%) were excluded because of missing values in one or more of the eight HUI3 domains. Of the 192 participants for whom full data were available, 170 (86.7%) were reassessed a mean of 1 year 2 months (SD 2mo) after the first assessment. Missing data were rarely observed (0.32% for the first assessment and 0.59% for the second). Missing data could not be imputed on the HUI3 because utility scores for each attribute consist of a single value.
Unlike the QOL Instrument, attributes on the HUI3 demonstrated a wide range of stability (Table II). High stability (G>0.9) was observed in ambulation and overall utility, with virtually no group change and only slight differences between individuals over time. Moderate stability (G=0.7–0.9) was observed in speech, vision, dexterity, cognition, and hearing. These scores demonstrated more substantial differences between individuals over time and a lower variability of scores between participants in comparison with attributes that achieved high stability. An exception to this trend was dexterity, which demonstrated only slightly less variability of scores between participants in comparison with ambulation and overall utility. Low stability was observed for pain (G=0.48) and emotion (G=0.24), which resulted from substantially lower variability in comparison with scores that achieved moderate or high stability.
Table II. Stability of scores on the Health Utilities Index Mark 3 (HUI3) over 1 year
Varianceexplainedby themaineffect oftime(not a pvalue)c
aVariability of scores. bDifferences between individuals over time. cGroup change over time. dGeneralizability coefficient (single measure of absolute agreement over 1y).
The cutoff points of 0.7 and 0.9 were selected because a stability of 0.7 over a short period (i.e. test–retest reliability) is recommended for making judgements regarding groups, and 0.9 is recommended for making judgements about individuals.21
Stability and variability
Scores on the QOL Instrument demonstrated an association between stability and variability (or ‘change’ and ‘range’), where domains with greater variance between participant scores showed higher stability. This trend was consistent across scores for Being, Belonging, and Becoming among self-reports, proxy-reports, and all participants (Table I).
An association between stability and variability was also observed for utility scores on the HUI3. Utility scores with higher stability demonstrated greater variability between participant scores than those with lower stability. This relationship is depicted graphically in Figure 2.
Two perspectives on quality of life
Correlations between scores on the HUI3 and QOL Instrument have been reported previously for Being (r=0.37), Belonging (r=0.17), Becoming (r=0.20), and Overall QOL (r=0.28).4 In the present study, these coefficients were corrected for lack of perfect stability (i.e. attenuation) over 1 year, which ranged from 0.55 to 0.70 for scores on the QOL Instrument among all participants (Table I) and 0.91 for overall utility scores on the HUI3 (Table II). Adjustments were made with the following formula:21
Disattenuated correlation coefficients between scores on the HUI3 and QOL Instrument were determined for Being (r=0.48), Belonging (r=0.35), Becoming (r=0.29), and Overall QOL (r=0.35), indicating that the two scales share up to 23% of the variance (i.e. [0.48]2). The modest increase from 14% to 23% of the shared variance indicated that the relationship between QOL and HRQOL was moderate at best, even when adjustments were made for imperfect stability over 1 year.
This study showed that measures of QOL and HROQL were moderately stable over 1 year among adolescents with CP. Scores on the QOL Instrument and HUI3 demonstrated virtually no group change over time, but there were substantial differences between individuals in terms of the magnitude and direction of change. Thus, less than perfect stability was due to participants’ changing in different directions and at different rates, rather than a systematic increase or decrease in scores across all participants (as illustrated conceptually in Figure 1b). The pattern of scores observed with the QOL Instrument and HUI3 was consequently similar to that depicted in Figure 1c. Scores with higher stability simply demonstrated fewer differences between individuals over time.
Relatively lower stability was observed in the domain of Becoming on the QOL Instrument (G=0.56) and among utility scores for emotion (G=0.24) and pain (G=0.48) on the HUI3. This suggests that these aspects of QOL or HRQOL are more variable from one time to another than are the other components of these constructs. Indeed, on a conceptual level, it should not be surprising at all to see that the participants’ perceptions of the future (i.e. Becoming) were not as stable over 1 year as the other scores. The same could be said of ratings of emotion and pain, which are inherently subjective and likely to be somewhat variable over time.
An important caveat to this conceptual argument is the fact that domains that demonstrated lower stability also had lower variability between participant scores (as illustrated with the theoretical example in Fig. 1d). In other words, almost all participants provided similar ratings for emotion and pain at the initial assessment (hence the low variability between scores), which meant that any changes observed 1 year later were relatively large. This measurement issue is important to bear in mind and speaks to the importance of using scales with appropriate psychometric properties.
The modest correlations between scores on the QOL Instrument and HUI3 suggests that the ‘overall’ and ‘on the surface’ perspectives, which we have referred to as QOL and HRQOL, are different dimensions among adolescents with CP. The strength of the relationship between these constructs remained modest at best, even when adjustments were made for a lack of perfect stability over 1 year (i.e. attenuation). We believe that QOL and HRQOL are complementary views of life experience and that one perspective is not inherently better than the other.5 We suspect that assessments of HRQOL might be more appropriate for evaluating the effects of medical or surgical interventions on biomedical status and activities of daily living.22,23 In contrast, measures of QOL might be useful for determining predictors of well-being or tracking changes within specific individuals over time to assess the impact of interventions designed to enhance life experiences of young people with disabilities.
The results of this study were limited in that the variance explained by the interaction between participant scores and time consisted of measurement error and true differences between individuals. Although we were unable to determine the relative size of each component, future studies could do so by examining the interaction between participant scores over short periods (such as 2 weeks) and a longer interval (such as 1 year, as in this study). The first value would consist almost entirely of measurement error, assuming that there is minimal true change in QOL or HRQOL over 2 weeks, and the second would consist of both measurement error and true differences. The approximate magnitude of true differences between participant scores over time could then be estimated by subtracting the first interaction term from the second.
Finally, future work should use construct validation methods to explore, systematically and prospectively, changes in QOL associated with known changes (known improvement or deterioration) in overall well-being of young people with disabilities. A knowledge of which aspects of QOL change and which are stable would contribute further to our understanding of the extent to which QOL can be considered a ‘state’ (a momentary perception of life experience that varies over time) or a ‘trait’ (something inherent to each individual).
What do these findings mean for Jack and his mother? At the very least, they suggest that the ‘on the surface’ and ‘overall’ perspectives are stable over 1 year. Jack’s ‘quality of life’ may increase or decrease over time, but such changes should not be attributed to the fact that Jack has CP. We should tell Jack’s mother that some adolescents with CP seem to improve over time and some worsen, but there is no apparent, systematic change in either direction. We should not be surprised to hear about the differences between the ‘on the surface’ and ‘overall’ views of life experience, because these are different ways of talking about QOL and should not be used interchangeably.
This work was supported by a grant for the Adolescent Study of Quality of Life, Mobility and Exercise (ASQME) from the Canadian Institutes of Health Research (MOP-53258). Members of the ASQME study group include Dr Doreen Bartlett, Dr Steven Hanna, Dr John Lawless, Dr Robert Palisano, Dr Dianne Russell, and Dr Stephen Walter. Barbara Galuppi was the expert study coordinator of ASQME. We tender special thanks to Dr Rebecca Renwick for discussing the conceptual background of the QOL Instrument, and to Mr Ted Myerscough for his technical assistance. The first author would like to thank his brother, Jack Livingston, and mother for their inspiration and support, and for consenting to be included in the vignette that appears in this report. The authors are also indebted to the adolescents and their families who so openly shared their thoughts and personal experiences throughout the ASQME.
List of abbreviations
Adolescent Study of Quality of life, Mobility and Exercise
Health-related quality of life
Health Utilities Index Mark 3
Ontario Motor Growth
Quality of life
Quality of Life Instrument for People with Developmental Disabilities