SEARCH

SEARCH BY CITATION

Keywords:

  • adolescents;
  • ethnicity;
  • Pacific;
  • quality of life;
  • utility weights

ABSTRACT

  1. Top of page
  2. ABSTRACT
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. References

Objectives:  Pacific Obesity Prevention in Communities (OPIC) is a community-based intervention project targeting adolescent obesity in Australia, New Zealand, Fiji, and Tonga. The Assessment of Quality of Life Mark 2 (AQoL-6D) instrument was completed by 15,481 adolescents to obtain a description of the quality of life associated with adolescent overweight and obesity, and a corresponding utility score for use in a cost–utility analysis of the interventions. This article describes the recalibration of this utility instrument for adolescents in each country.

Methods:  The recalibration was based on country-specific time trade-off (TTO) data for 30 multiattribute health states constructed from the AQoL-6D descriptive system. Senior secondary students, in a classroom setting, responded to 10 health state scenarios each. These TTO interviews were conducted for 24 groups, comprising 279 students in the four countries resulting in 2790 completed TTO scores. The TTO scores were econometrically transformed by regressing the TTO scores upon predicted scores from the AQoL-6D to produce country-specific algorithms. The latter incorporated country-specific “corrections” to the Australian adult utility weights in the original AQoL.

Results:  This article reports two methodological elements not previously reported. The first is the econometric modification of an extant multi-attribute utility instrument to accommodate cultural and other group-specific differences in preferences. The second is the use of the TTO technique with adolescents in a classroom group setting. Significant differences in utility scores were found between the four countries.

Conclusion:  Statistical results indicate that the AQoL-6D can be validly used in the economic evaluation of both the OPIC interventions and other adolescent programs.


Introduction

  1. Top of page
  2. ABSTRACT
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. References

The Pacific Obesity Prevention in Communities (OPIC) project is a four-country project funded in Fiji and Tonga by the Wellcome Trust, New Zealand by the Health Research Council, and Australia by the National Health and Medical Research Council to expand the capacity of the Pacific region to respond to the obesity crisis. The region is faced with among the highest rates of obesity in the world. Prevalence rates for overweight and obesity are around 75% in Tonga [1] and 80% for the Pacific populations living in New Zealand [2,3]. The impact of obesity as a risk factor for diseases such as heart disease, stroke, diabetes, selected cancers, and osteoarthritis has been well documented. A World Health Report in 2002 [4] estimated that obesity, which was the 10th leading cause of avoidable burden, would be the seventh leading cause for 2010 and 2020.

The limited capacity of the Pacific Region to respond to the obesity epidemic and the poor evidence base of what works in terms of obesity prevention were the key factors underpinning the project [5]. The OPIC project set out to address these two issues through the development of comprehensive, community-based intervention programs which targeted adolescents (aged 12–18 years) in each of the four countries. A quasi-experimental design was employed with an intervention period of 3 years and a cohort follow-up, and changes in body mass index as the primary outcome variable.

The linked economic studies included the administration of a health-related quality-of-life (QoL) measure to both facilitate description of the QoL burden of adolescent overweight and obesity, and as an outcome measure in a cost–utility analysis (CUA) of the interventions. The latter will enable a comparison of the efficiency of the obesity interventions implemented against a broader spectrum of health-care interventions.

Measuring QoL for Economic Evaluation

Before the development of CUA, economic evaluation of health services either ignored QoL or treated QoL as an “intangible” that could be noted and described, but not quantified or included as an integral part of the health outcome. CUA has attempted to overcome this deficit by adopting the quality-adjusted life-year (QALY) as the unit of output for health benefits in cost-effectiveness studies [6]. One of two approaches has been adopted.

First, in a “holistic” or scenario-based approach to measurement, the health states relevant to the evaluation of a health program are described in a series of scenarios. These are then rated using a scaling device such as the time trade-off (TTO) or standard gamble (SG) to obtain a “utility” index, an index of the strength of a person's preference for a health state [6]. The index is then used to obtain QALYs. The construction of the health scenarios and the rating exercise both require surveys. Normally, patients who have experienced the health states are consulted for scenario construction, and a random population sample is used for rating them.

The second “decomposed” approach employs multiattribute utility (MAU) theory [7] requires the preliminary construction of a generic MAU QoL instrument which is capable of describing numerous health states and assigning a utility score to each of these [8]. MAU instrument construction requires the creation of a descriptive system describing multiple health states. This involves the decomposition of a health state into multiple dimensions of health, which are described by one or more “items,” that is, a series of questions, each with multiple responses, which describe the dimension and the intensity of the health state. Generic instruments usually purport to include all significant dimensions of health. To convert the descriptive instrument into a MAU instrument, a “scaling” system is created which is capable of assigning utility scores to every combination of the instrument's health states. This requires the calibration (scaling) of item responses and the decomposition of the dimensions into holistic health states. Literally, the MAU approach decomposes health states, assigns utility scores to the decomposed parts, and then recombines the parts using an appropriate model to determine an overall utility score. The attraction of the MAU instrument is, inter alia, that it obviates the need for the two surveys required by the holistic approach and it allows for the continuous collection of data in longitudinal studies.

The final MAU instrument is a questionnaire similar in format to a number of disease-specific and psychometric instruments, however, differing in two respects. First, the “descriptive system”—the questionnaire—is generic, which purports to cover all health states (a property also claimed by a small number of psychometric instruments including the SF36). Second, the instrument's scoring system purports to measure “utility,” the strength of people's preferences and in a way which gives the instrument an “interval” property. A numerical increment (e.g., 0.2) represents the same improvement in QoL anywhere on the scale. For example, an increment from 0.3 to 0.5 is the same incremental improvement according to some external criterion as a move from 0.7 to 0.9.

The strength of CUA for economic evaluation is derived from this latter property. In principle, every health state or health state improvement can be described and measured on the same scale, and, consequently, disparate health program interventions can be evaluated on a “level playing field.” In particular, increments to the quality and quantity of life can be compared. The all-important interval property is obtained from the “scaling” instrument. While five such instruments (SG, TTO, person trade-off, rating scale, and magnitude estimation) have been used [9], the first two are the most widely used. The TTO is used in the present study. During a structured interview, respondents (study population or the general public) are asked what proportion of an assumed life expectancy they would be prepared to sacrifice to be in full health rather than in the health state being evaluated. With a zero rate of time preference, an answer of 50%, 20%, and 5%, respectively, therefore indicates a QoL index of 0.5, 0.8, and 0.95 on a 0 to 1 scale, where 0 and 1 represent death and full health, respectively.

In principle, an instrument should only be used in a population for which the instrument has been “validated”—successfully tested, usually by comparison with the results from another instrument which has been independently validated. The greater the difference between the population in which the instrument is to be used and the initial population from which it was created, the greater the likelihood that the instrument will not correctly measure population preferences. For this reason, instruments should not be used without independent evidence of validity.

Two Australian MAU instruments have been created, namely the Assessment of Quality of Life (AQoL) [10,11] and the Assessment of Quality of Life Mark 2 (AQoL-6D) [12]. By mid-2010, a third instrument, AQoL-8D, will be available on the Web (http://www.aqol.com.au). The AQoL-6D instrument is an adaptation of the AQoL, designed to increase sensitivity to health state variations close to normal health and to extend the coverage of AQoL-6D. Therefore, while AQoL has four dimensions, AQoL-6D has six dimensions, viz., independent living, social relationships, physical senses, psychological well-being, pain, and coping (Fig. 1). Both these instruments were scaled using a sample of the Australian population representative of the socioeconomic profile of adult Australians.

image

Figure 1. Structure of AQoL-6D.

Download figure to PowerPoint

In contrast with some instruments, both the AQoL-4D and AQoL-6D were conceptualized in terms of handicap: poor health is described in terms of its impact upon people's capacity to carry out normal activities rather than the effect upon a person's impairment or disability (so-called within-the-skin descriptive systems).

The use of the AQoL-6D (or any other existing MAU instrument) in the OPIC project was considered problematical, as its utility weights were calculated from the health state preferences of Australian adults. In contrast, the target population of the OPIC study was adolescents in Australia, New Zealand, Fiji, and Tonga, and it was deemed unlikely that their utility values would be the same as those of Australian adults. A review of the published literature suggests significant cultural variations in health state preferences [13–18]. Adolescents are also likely to value their health differently to adults, given their social values, support structures, lifestyles, and experience. It was therefore important to use adolescent rather than adult values, and, more specifically, country-specific adolescent values in the OPIC study. Therefore, the utility weights were recalculated and validated for each of the four countries using the adolescents' survey results from each site.

Methods

  1. Top of page
  2. ABSTRACT
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. References

Because of the diversity of health states which were likely to be encountered, and high cost of using the scenario-based, holistic method of utility measurement, the OPIC protocol employed the decomposed MAU methodology. Because of the need for instrument sensitivity to near-normal health state and a handicap-based conceptualization of health, the AQoL-6D was adopted as the “base instrument”[12]. It consists of 20 items grouped into six dimensions, each of which is separately modeled and then combined to obtain a single AQoL-6D utility score.

The AQoL-6D was created with a three-part calibration that allows a relatively easy recalibration of the instrument. As described below, in the first two parts, multiplicative models were used to determine, respectively, algorithms for the dimension scores and for a total multidimensional—MA—score using the TTO values obtained for item responses and dimensions. In the third part, the latter score was adjusted econometrically to offset the potential effects of “structural or preference dependence,” which could result in “double counting” of the disutility of some dimensions. To achieve this, multiattribute (MA) scenarios were independently constructed, evaluated using the TTO methodology, and used as the dependent variables in a regression in which total AQoL scores and country-specific demographic factors were the independent variables. Results were used as the stage 3 adjustment. In the present study, this third stage was replicated using site-specific TTO scores for MA scenarios, which were constructed to be of most relevance for obese youth.

The process of adaptation involved the six stages shown in Figure 2. These were: 1) adaptation of the AQoL descriptive system; 2) preparation of 30 MA scenarios for assessment using the TTO scores; 3) development of a protocol and proforma for the classroom-based use of the TTO; 4) administration of the TTO and “debriefing”—qualitative assessment of the understanding and difficulty of the TTO task; 5) data collection and editing; and 6) the econometric recalibration of the AQoL-6D utility weights. Results were used in a description of the full OPIC project populations using the utility scores produced from the adapted four country-specific AQoL-6D scoring algorithms.

image

Figure 2. Steps in the methodology of the Obesity Prevention in Communities quality-of-life assessment.

Download figure to PowerPoint

Adaptation of the AQoL-6D Descriptive Instrument

In each country, focus groups of adolescents were conducted to determine semantic and cultural equivalence of the instrument, that is, to determine whether phrases, examples, or words used in the descriptive system had clear and equivalent meaning in the different study groups, and, when appropriate, to determine replacements from the local idiom. In Tonga, the AQoL descriptive system was double translated into the Tongan language.

Preparation of the Health State Scenarios

The AQoL-6D questionnaire asks 20 questions about a person's health, which are categorized into six dimensions. Stage 2 of the methodology used combinations of potential responses from the 20 questions to make 30 health state scenarios which participants were asked to value in comparison with instrument-best health (Table 1). In principle, these would have been constructed using an experimental design to achieve efficiency. In practice, this resulted in absurd combinations of health states—bedridden, depressed, but always full of energy—reflecting the fact that structural independence is not achievable with a handicapped base descriptive system. Consequently, an ad hoc approach was adopted which ensured that, while maximizing simplicity of the health states, all relevant combinations of dimensions were included in one or more of the scenarios (Table 2). An example of a scenario is given in Figure 3.

Table 1.  Multiattribute time trade-off health states
Multiattribute utility (MAU) time trade-off (TTO) health statePhysical abilitySocial and family relationshipsMental healthCopingPainVision, hearing, and communication
 1I can do jobs around the house (such as cleaning my room or helping with meals) only very slowly unless I have help. I have a lot of difficulty getting around by myself outside my house. Walking is difficult for me (I walk short distances only. I have difficulty walking upstairs). Many tasks like going to the toilet, washing myself, getting dressed, or eating are difficult, and I need help to do them.My close friendships make me generally happy. My relationship with my family is not affected by my health. My ability to participate in group activities is not affected by my health.ExcellentExcellentExcellentExcellent
 2ExcellentExcellentI sometimes feel worried. I sometimes feel sad. I am usually calm. I occasionally feel in despair (lost and hopeless).I am occasionally full of energy. I mostly feel that I manage my life well. I mostly feel I can cope with life's problems (such as conflict with family or friends, doing exams, etc.).ExcellentExcellent
 3I can do jobs around the house relatively easily without help. I have moderate difficulty getting around by myself outside my house. I find walking or running slightly difficult. I have no real difficulty with tasks such as going to the toilet, washing myself, getting dressed, or eating.ExcellentExcellentExcellentI have moderate pain. I experience serious physical pain less than once a week. Pain sometimes interferes with my usual activities.Excellent
 4ExcellentExcellentExcellentI only occasionally feel that I manage my life well. I partly feel I can cope with life's problems (such as conflict with family or friends, doing exams, etc.).I suffer from severe pain. I experience serious physical pain less than once a week. Pain sometimes interferes with my usual activities.I have some difficulty focusing on things, or I do not see them sharply. I have difficulty hearing things clearly, and often I do not understand what is said.
 5ExcellentThere are some parts of my relationship with my family that are affected by my health. There are some group activities I am not involved in because of my health.I sometimes feel sad. I am sometimes calm, sometimes agitated.ExcellentExcellentI have some difficulty being understood by people who do not know me, but I have no trouble understanding what others are saying to me. I see normally.
 6ExcellentMy close friendships make me neither happy nor unhappy. There are many parts of my relationship with my family that are affected by my health.I often feel worried. I sometimes feel sad.I only occasionally feel that I manage my life well. I am usually tired and lacking energy.I suffer from severe pain. I experience serious physical pain three to four times a week. Pain often interferes with my usual activities.Excellent
 7I have a little difficulty getting around by myself outside my house. I find walking or running slightly difficult (I cannot run to catch a bus or train, I find walking uphill difficult).ExcellentI rarely feel sad. I am usually calm.ExcellentI have moderate pain. I experience serious physical pain less than once a week. Pain rarely interferes with my usual activities.I have difficulty hearing things clearly, and often I do not understand what is said. I am understood only by people who know me well, and I have great trouble understanding what others are saying to me.
 8I have great difficulty walking, and I cannot walk without a walking stick or frame or someone to help me.ExcellentI sometimes feel worried. I usually feel sad.I am usually tired and lacking energy. I partly feel I can cope with life's problems (such as conflict with family or friends, doing exams, etc.). ExcellentExcellent
MAU TTO health statePhysical abilitySocial and family relationshipsMental healthCopingPainVision, hearing, and communication
 9I have a little difficulty getting around by myself outside my house. I find walking or running slightly difficult (I cannot run to catch a bus or train, I find walking uphill difficult).My close friendships make me generally happy. There are some group activities I am not involved in because of my health.ExcellentExcellentI have moderate pain. I experience serious physical pain less than once a week. Pain rarely interferes with my usual activities.Excellent
10I have great difficulty walking, and I cannot walk without a walking stick or frame, or someone to help me.There are many parts of my relationship with my family that are affected by my health. There are many group activities that I am not involved in because of my health.ExcellentI feel I can cope very little with life's problems (such as conflict with family and friends, doing exams, etc.). I only occasionally feel that I manage my life well.ExcellentI have a lot of difficulty seeing things, my vision is blurred, and I see just enough to get by. I hear very little indeed. I cannot fully understand loud voices speaking directly to me.
11ExcellentExcellentExcellentI am usually full of energy. I mostly feel I can cope with life's problems (such as conflict with family or friends, doing exams, etc.).I have moderate pain. I experience serious physical pain less than once a week.I have some difficulty hearing, I have trouble hearing softly spoken people or when there is background noise. I have no trouble speaking to others or understanding what they are saying.
12ExcellentMy close friendships make me generally unhappy. There are many group activities that I am not involved in because of my health.I often feel worried. I am sometimes calm and sometimes agitated.ExcellentExcellentI see normally. I hear normally.
13ExcellentThere are some parts of my relationship with my family that are affected by my health. There are some group activities I am not involved in because of my health.I often feel in despair (lost and hopeless). I usually feel sad.I sometimes feel that I manage my life well. I partly feel that I cope with life's problems (such as conflict with family and friends, doing exams, etc.).I have moderate pain. I experience serious physical pain very rarely.Excellent
14I cannot do most jobs around the house unless I have help. I have a lot of difficulty getting around by myself outside my house. I have great difficulty walking, I cannot walk without a walking stick or frame, or someone to help me. Many tasks like going to the toilet, washing myself, getting dressed, or eating are difficult, and I need help to do them.ExcellentI often feel worried. I am sometimes calm, sometimes agitated.ExcellentI have moderate pain. I experience serious physical pain less than once a week.I see normally. I have no trouble speaking to others or understanding what they are saying.
15I can do jobs around the house (like cleaning my room or helping with meals) relatively easily without help. I have a little difficulty getting around by myself outside my house. I have no real difficulty with walking or running.ExcellentI sometimes feel in despair (lost and hopeless). I usually feel sad.I am usually tired and lacking energy. I sometimes feel that I manage my life well.ExcellentExcellent
16I cannot do most jobs around the house unless I have help. I have a lot of difficulty getting around by myself outside my house. I have great difficulty walking, and I cannot walk without a walking stick or frame, or someone to help me. Many tasks like going to the toilet, washing myself, getting dressed, or eating are difficult, and I need help to do them.My close friendships make me neither happy nor unhappy. Some parts of my relationship with my family are affected by my health.ExcellentExcellentI suffer from severe pain. I experience serious physical pain three to four times a week.Excellent
17I can do jobs around the house relatively easily without help. I have no real difficulty with walking or running. I have a little difficulty getting around by myself outside my house.My close friendships make me generally unhappy. There are some group activities that I am not involved in because of my health.ExcellentI am occasionally full of energy. I feel I can mostly cope with life's problems (such as conflict with family or friends, doing exams, etc.).ExcellentI hear very little indeed (I cannot fully understand loud voices speaking directly to me). I am understood only by people who know me well, and I have great trouble understanding what others are saying to me.
18ExcellentMy close friendships make me neither happy nor unhappy. There are many parts of my relationship with my family that are affected by my health. There are many group activities I am not involved in because of my health.ExcellentExcellentExcellentI have a lot of difficulty seeing things, my vision is blurred. I can see just enough to get by. I hear very little indeed, I cannot fully understand loud voices talking directly to me. I am understood only by people who know me well, and I have great trouble understanding what others are saying to me.
19I have no real difficulty with walking or running.Some parts of my relationship with my family are affected by my health.ExcellentI am occasionally full of energy. I sometimes feel that I manage my life wellExcellentI see normally. I hear normally. I have some difficulty being understood by people who do not know me. I have no trouble understanding what others are saying to me.
20ExcellentMany parts of my relationship with my family are affected by my health.I sometimes feel in despair (lost and hopeless). I often feel worried. I usually feel sad. I am usually agitated.I am usually tired and lacking energy. I only occasionally feel that I manage my life well. I partly feel I can cope with life's problems (such as conflict with family or friends, doing exams, etc.).ExcellentExcellent
21I have a lot of difficulty getting around by myself outside my house. I find some tasks such as going to the toilet, washing myself, getting dressed, or eating difficult, but I manage to do them on my own.Some parts of my relationship with my family are affected by my health. There are many group activities I am not involved in because of my health.ExcellentI only occasionally feel that I manage my life well.I have moderate pain.Excellent
22ExcellentExcellentExcellentI am occasionally full of energy. I mostly feel that I manage my life well. I feel I can cope very little with life's problems (such as conflict with family or friends, doing exams, etc.).I have moderate pain. I experience serious physical pain less than once a week.I have some difficulty hearing or I do not hear clearly (I have trouble hearing softly spoken people or when there is background noise). I have some difficulty being understood by people who do not know me. I have no trouble understanding what others are saying to me.
23I can do jobs around the house relatively easily without help. I have no difficulty getting around by myself outside my house. I have no real difficulty with walking or running.ExcellentI occasionally feel worried. I rarely feel sad.I am occasionally full of energy. I sometimes feel that I manage my life well.ExcellentExcellent
24I find walking or running slightly difficult (I cannot run to catch a bus or a train, I find walking uphill difficult).ExcellentI occasionally feel worried. I rarely feel sad.I sometimes feel that I manage my life well. I partly feel I can cope with life's problems (such as conflict with family or friends, doing exams, etc.).I have moderate pain.Excellent
25I have no real difficulty with walking or running.Some parts of my relationship with my family are affected by my health.I usually feel sad.ExcellentI experience serious physical pain three to four times a week. Pain often interferes with my usual activities.Excellent
26ExcellentExcellentI often feel in despair (lost and hopeless). I occasionally feel worried. I usually feel sad.I sometimes feel that I manage my life well.I have moderate pain. Pain often interferes with my usual activitiesI have some difficulty hearing or I do not hear clearly (I have trouble hearing softly spoken people or when there is background noise).
27ExcellentMany parts of my relationship with my family are affected by my health.ExcellentI mostly feel that I manage my life well.I have moderate pain. I experience serious physical pain less than once a week.I hear normally.
28I have moderate difficulty getting around by myself outside my house.My close friendships make me generally unhappy.I sometimes feel in despair (lost and hopeless). I usually feel sad. I am usually agitated.ExcellentExcellentExcellent
29I have a little difficulty getting around by myself outside my house. I find some tasks such as going to the toilet, washing myself, getting dressed, or eating difficult, but I manage to do them on my own.My close friendships make me generally unhappy.I often feel in despair (lost and hopeless).I feel I can cope very little with life's problems (such as conflict with family or friends, doing exams, etc.).I experience serious physical pain three to four times a week.Excellent
30I have moderate difficulty getting around by myself outside my house. I find walking or running slightly difficult (I cannot run to catch a bus or a train, I find walking uphill difficult). I find some tasks such as going to the toilet, washing myself, getting dressed, or eating difficult, but I manage to do them on my own.There are many group activities I am not involved in because of my health.I sometimes feel worried. I usually feel sad. I am usually agitated.ExcellentI have moderate pain.Excellent
Table 2.  Time trade-offs: participation and responses
 AustraliaFijiTongaNew ZealandTotal
No. groups conducted666624
No. students participating68708160279
No. students completing     
 Sort 1 (10 scenarios)2124282497
 Sort 2 (10 scenarios)2622261286
 Sort 3 (10 scenarios)2124272496
Total responses completed6807008106002790
No. responses invalid32106
Total valid responses6776988096002784
image

Figure 3. Example of time trade-off performa.

Download figure to PowerPoint

Development of the TTO Proforma

The TTO protocol adopted was an adaptation of the methods used extensively at the Monash Centre for Health Economics, and described by Iezzi and Richardson [19]. As noted by Tilling et al. [20], health states worse than death are often unsatisfactorily described. The Monash protocol for evaluating these states is described by Richardson et al. [21]. Adaptations for classroom were developed using a collaborative process of design, evaluation, testing, critique, and discussion. A sample of the final graphics is attached as Figure 3. Input was sought from a psychologist, health economists, and graphic designers. Different designs for the materials were piloted with both adults and adolescents unfamiliar with the TTO process. The result was a clear, simple, and easy-to-use design with a modern, clean format adapted from the materials used during the construction of the original AQoL-6D. The materials and process were pilot tested with Australian secondary students.

Administration of the TTO Interviews

A sample of 60 students was required from each of the four OPIC sites. Based on the completion of 10 scenarios per student, this would result in a total of 600 scenarios per site, sufficient responses to facilitate prediction of the new utility weights with 95% confidence.

TTOs are normally completed by an interviewer with a single respondent. For cost and logistical reasons, this was not practical in the OPIC project. The interviews were administered by two trained facilitators in a class setting to groups of 10 to 15 students, all of whom gave individual responses. Six groups were conducted in each of the four countries. The exercise was completed only by senior students because of the cognitive complexity of the task. The time frame was generally one classroom period.

The students were told to imagine that the health state described in a scenario (“health state A”) would continue for 10 years followed by death. Having “immersed” themselves in the health state, they were then asked whether they would be prepared to live for less time if they could move to health state B—excellent health where all the dimensions were at their best possible level. They were required to specify the amount of time in excellent health they considered equal in value or desirability to spending the rest of their life (10 years) in health state A. In the example (Fig. 3), if the respondent marked 4 years on the bar, this would suggest that they were prepared to trade-off 6 years of their remaining life of 10 years to enjoy 4 years in full health.

Several examples were worked through as a group beforehand, and students were allowed to ask questions. Then, each of the scenarios was read aloud by the facilitator, and students were given time to consider their response before moving to the next scenario. Assistance was provided to individual students who had difficulty understanding the instructions. Two facilitators moved among the students checking responses, and asked for explanations where students had given extreme responses (trading off all or no time) to ensure that they understood the meaning of their answer. In Tonga, the sessions were conducted in English as it is the primary language of instruction. Nevertheless, a Tongan-speaking member of the research team was in attendance to provide clarification in Tongan where necessary.

The TTOs were conducted in a very quiet, controlled classroom environment. A classroom teacher was in attendance at all times, in addition to the two facilitators. The students were not permitted to talk to each other, nor to show or share their responses. The size of the groups was necessarily restricted to ensure that facilitators could check the understanding of individual students. Later debriefing suggested that students understood and approached the task in a serious manner, and had responded in a considered and rational manner.

Recalibration of the Utility Weights

The competed health state utility scores were used to recalibrate the AQoL-6D scaling system. This used the three-stage procedure described in more detail below. TTO data from the OPIC study were used to replace the original data used in stage 3.

As noted, stages 1 and 2 of the AQoL-6D procedure use multiplicative models to obtain scores, first for the AQoL-6D's six dimensions and, second, to obtain a single score that combines the dimensions. Multiplicative models are recommended by decision analytic theory when importance weights sum to more than unity [22] and have been used previously in the AQoL-4D and Health Utility Index 1, 2, and 3 [23]. The model is very similar to the simplified model in Equation 1 in which U* is the final instrument utility, which is the product of the utility scores for each of the six dimension scores, Ui, that is, the dimension scores are mapped multiplicatively into a single score. In this simplified equation when any item utility is zero, U* is zero. With all item utilities equal to 1.00, U* is also equal to 1.00. That is, utility is measured on a 0 to 1 scale. A given percentage reduction in any dimension utility will cause the same percentage reduction in overall utility, U*, at any level.

  • image(1)

Equation 1, however, is too simple: each of the utilities has equal importance and if any item is zero the QoL of other aspects of health is unimportant. For this reason, the multiplicative model recommended by Decision Theory was used [22]. This takes the following form:

  • image(2)

where DU is instrument disutility, DUi is the measured disutility from dimension i, wi is the importance weight of dimension i, and k is the scaling constant described as:

  • image(3)

Despite its apparent complexity, Equation 2 is essentially the same as Equation 1 except that each of the dimensions has a unique importance weight wi attached to it. The model is also more easily expressed using DU as the metric (where DU = 1.00 − U), as this avoids the complication of using negative scores for states worse than death). Thus, Equation 2 essentially maps importance weighted dimension disutilities into a single disutility score. From Equation 2, DU also varies from 0 to 1.00, which can be seen by setting all DUi = 0 and all DUt = 1 respectively.

In a simple additive model (DU = w1+ . . .  + w6D6), item weights must sum to unity (Σiwi = 1). The analogous requirement for the multiplicative model is given in Equation 3 from which the scaling constant, k, may be derived.

Decision Theory requires that, inter alia, dimensions be structurally independent (orthogonal): a single “element” of a poor health state should not contribute to the disutility score of two or more items [22]. For example, pain associated with an illness that is independently measured by a pain item or dimension should not affect the score for mobility, social relationships, and psychological well-being. If this occurs (as it commonly does), then the impact of pain upon final utility will be exaggerated (double counting). This type of structural dependence is virtually unavoidable in all, but the most simplistic models of health.

For this reason, the AQoL-6D used an econometric “correction” or adjustment. A number of MA health state scenarios were included in the TTO survey. These were used to calibrate an econometric model in which the MA health state was “explained” by the AQoL-6D scores. In this final model, “double counting” of elements will not result in an exaggerated utility score as the scores are constrained by the left-hand side MA values. In the present project, the MA-TTO values used in this third stage were replaced with the MA-TTO values obtained in each of the four countries as described above. The following econometric model was used to make the correction:

  • image(4)

where TTO_DU is the disutility computed from the TTO scores for health states, and DUAQoL-6D is the disutility of the overall AQoL-6D score. Equation 4 was estimated using both panel data techniques with individual observations and also OLS regressions using average values for the health states. This equation is used to obtain the new utility weights for the four countries. (These disutilities can be converted into utilities using the formula U = 1 − DU.) In the above equation, α is the coefficient to be estimated for jth individual in ith health state for country C (i.e., Australia, New Zealand, Fiji, and Tonga), and ε is the error term which comprises of an individual-specific effect, a health state-specific effect, and an additional idiosyncratic term.

To compute the QoL scores from the final calibrated instrument, individual item responses from the survey population were first converted into item scores which were then converted into dimension scores which were further combined into an overall stage 2 AQoL-6D disutility score. The model in Equation 4 was then used to estimate utility scores. This was done for each of the 15,481 individuals in the OPIC survey.

Results

  1. Top of page
  2. ABSTRACT
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. References

Results from the TTO interviews are summarized in Table 3. Twenty-four groups and 279 students were interviewed, which resulted in a total of 2790 scenario evaluations or “observations.” Because interviews were conducted in schools and during school hours, there was a 100% response rate. Few students found the task taxing, and only six responses were rated as invalid (generally because the point marked on the scale was not decipherable or enclosed a range of values). The target number of 20 completed responses for each of the 30 scenarios was achieved and exceeded, particularly in Tonga where 81 students (rather than the targeted 60) participated in the interviews.

Table 3.  Responses with extreme values by site
No. responses with extreme valuesAustraliaFijiTongaNew Zealand
Value of 0 (would rather die)0095
Value of 10 (not prepared to trade-off any time)7111013

There was significant variation in the scores for each of the MA states which is normal in value elicitation surveys. This commonly reflects a misunderstanding of a particular question or difficulty with the framing of the question, although assessment of comprehension suggested that this was high in all of our groups. This did not, however, have a large effect.

Despite this, the average scores per MA state in each country were less varied. As shown in Tables 3 and 4, most of the mean utility values were in the range 0.5 to 0.8. At the lower end of the range, this result is plausible. A utility of 0.5 indicates that a person will give up 50% of their life to avoid the health state, and it is unsurprising that few states had values less than this. In the upper range, there are few health states close to normal health. This reflects the design of the MA—scenarios which included multidimensional, poor health which were sufficiently severe to attract attention and consideration.

Table 4.  Frequency of average multiattribute utility scores (adjusteda) within specified ranges
RangeAustraliaFijiNew ZealandTonga
No.%No.%No.%No.%
≥8.000.026.713.300.0
7.0–7.9930.026.713.313.3
6.0–6.91033.31240.0310.0826.7
5.0–5.91033.31033.31033.3930.0
4.0–4.913.3413.3413.3723.3
3.0–3.900.000.0930.0413.3
<3.000.000.026.713.3
Total30100.030100.030100.030100.0

Table 5 reports results of the econometric analyses for the four countries using the functional form of Equation 4. For each country, the model in Equation 4 was used to analyze individual-level data using maximum likelihood estimation of panel data, and the mean-level data for the 30 MA-TTO health states using OLS regression methods. The calculation of utility scores is explained in Box 1.

Table 5.  Dependent variable: multiattribute time trade-off disutility scores by country
RegressionIndividual data* model MLE panelMean data* model OLS
  • *

    Regression coefficient Z score.

  • Individual E-type data.

  • Mean data for E-type health states.

DUAQoL6D1.44 (13.16)1.19 (13.0)
AustraliaWald chi-square 173Wald chi-square 168
Log-like −798Log-like −15.3
n = 677R2 = 0.95
 n = 30
Fiji1.31 (13.18)1.57 (18.46)
Wald chi-square 173Wald chi-square 174
Log-like −668Log-like −13.7
n = 698R2 = 0.85
 n = 30
New Zealand1.01 (14.3)0.87 (13.5)
Wald chi-square 203Wald chi-square 183
Log-like −591Log-like −4.4
n = 600R2 = 0.86
 n = 30
Tonga1.38 (13.52)1.13 (14.60)
Wald chi-square 182.91Wald chi-square 213.02
Log-like −790Log-like −2.15
n = 809R2 = 0.80
 n = 30

Box 1. Calculating Utility Scores.

Using the mean results from Table 5, the utilities for the four countries are calculated as UA = (1 − DU1.19); UF = (1 − DU1.59); UNZ = (1 − DU0.87), and UT = (1 − DU1.13), respectively, where UA, UF, UT, and UNZ are the utilities for Australia, Fiji, New Zealand, and Tonga, and DU is the disutility obtained by inserting item responses into the AQoL-6D algorithm (available on the Web site: http://www.aqol.com.au).

If the initial, unadjusted utility was 0.7, then the algorithms above would result in utilities of 0.76, 0.85, 0.65, and 0.74, respectively.

The model estimates for individual data and mean data for each country reveal that the structure of health state valuations differs in the four geographical locations. Results for both the individual and mean analysis are statistically significant. As most of the variation in these models is around the mean of the 30 MA variables, the single predicted score from the model for the 30 MA health states is considered to provide a better estimate. Thus, while individual data also produce significant parameter estimates, they cannot explain variation in individual scores around a single health state. The explanatory power of the equation for the between-health state variation is therefore better indicated by the analysis of mean, not individual data. It is also the more relevant result, as the economic evaluations which will use the instruments are generally based on mean, not individual, estimates of the QoL.

Figures 4–7 plot the MA-TTO mean scores for each health state against the average of the MA-TTO scores predicted from the econometric model for each country. In each of these graphs, the scatterplot for the observations and the corresponding regression fit is displayed. For all four countries, the regression fit coincides with the 45 degree line passing through origin, which suggests that the TTO scores predicted from the model in Equation 4 provide a good predictor of the actual TTO scores. Table 5 reports the results for regression of actual TTO scores on predicted scores from estimating the model in Equation 4 for the mean analyses. The coefficient of the TTO score is statistically significant for all the four countries, and the coefficient of the constant term is not statistically significant. The R2 coefficient in the final column indicates that, considering the various possible sources of error in the procedures described above and the limited range of the MA states, the regression's explanatory power is very good for three of the four countries. The low R2 in Tonga is primarily attributable to the clustering of the data shown in Figure 7. No values exceed 0.6. Nevertheless, the results indicate that there is no systematic bias in the estimated transformation.

image

Figure 4. Mean analysis for 30 health states: Australia.

Download figure to PowerPoint

image

Figure 5. Mean analysis for 30 health states: New Zealand.

Download figure to PowerPoint

image

Figure 6. Mean analysis for 30 health states: Fiji.

Download figure to PowerPoint

image

Figure 7. Tonga mean analysis: 30 health states.

Download figure to PowerPoint

Discussion

  1. Top of page
  2. ABSTRACT
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. References

In principle, each socioeconomic and age cohort in every culture may have different values and preferences. Every comparative study, therefore, involves a degree of oversimplification, and this problem increases as cultural differences increase. A second and related problem is that MAU instruments are imperfect. The range of items included is limited and may exclude elements relative to a particular culture. The conclusion is, again, that measurement is imperfect. But, it was acknowledgment of this that drove the present methods. The AQoL-6D was, in the authors' judgment, the most sensitive instrument for the study subject. Adaptation of the AQoL, as described, was second best to the prohibitively expensive creation of new instruments in each country. Nevertheless, one test of our success was the response to the questions in the different countries, and this was encouraging.

While it is acknowledged that the views of adolescents about the prospect of living in either perfect or imperfect health are likely to differ from those of adults with more life experience, we are confident that these older adolescents understood the concept of trading years of life, treated the task seriously, and were capable of making considered judgments. Contrary to expectation, few of the students appeared to have difficulty with the notion of trading quantity and QoL. This is indicated in Table 3 by the very small number of students who opted for extreme (no trade) options. Tonga and New Zealand were the only sites where respondents opted for death rather than time in a morbid health state. Nevertheless, the numbers involved—9 and 5—represented only 1.1% and 0.6% of responses in the two sites, respectively. While it is acknowledged that the checking of student responses may have influenced the relatively low number of ceiling and floor responses, the facilitators were not aware of any respondents changing their proposed extreme responses as a result of questioning by the facilitators. Adolescents were able to provide considered and reasoned responses when questioned about their choice of extreme values. Those who gave scores of 10 would typically respond that life was too precious, and they were not prepared to trade regardless of the severity of the health state. The results obtained were plausible as were the variations between countries. To that extent, we are confident that data represent real differences in health state preferences between adolescents in the four sites.

Comments made by the respondents supported the ranking of the results by country. Anecdotally, students of Pacific origin often expressed concern about the burden which a particular health state would place on their families: “It would not be fair on my family to live like that.” No such comments were forthcoming from Australian students, and this is reflected in their generally higher mean scores. Pacific Island children (including children of Pacific Island origin living in New Zealand) typically live in larger families/households. They are more likely to have experienced living with a chronically ill family member, and know the impact, burden, and responsibility it places on the family unit.

Just as health state preferences will vary between cultures, they are likely to vary between demographic groups within a country. Economic evaluations of pediatric interventions have traditionally used adult health state valuations. Nevertheless, while very little work has been undertaken around the elicitation of values from adolescents themselves, there is a growing body of literature that acknowledges the need for the development of age-appropriate generic utility instruments and the elicitation of preferences from children themselves rather than the use of adult values. There is also an acknowledgment that adolescents are sufficiently sophisticated to use such valuation methods and that their values may differ significantly from those elicited from adults given different attitudes toward risk [24–26].

The (individual) adjustment models were used in conjunction with AQoL-6D to estimate QoL scores for the full OPIC population of 15,481 by inserting self-completed AQoL-6D scores into the three-part scoring algorithm for each country. Details and other results from the OPIC study are reported elsewhere. Nevertheless, the summary results provide both a summary of the relative QoL estimates from the four countries and a test of the discriminatory power of the resulting instruments in the context of youth obesity. The numerical values of the scores cannot, however, be compared directly with other TTO results, without some adjustment. TTO utility scores are measured on a 0.00 to 1.00 scale, where 1.00 is the best state which may be described as “good,”“excellent,”“best imaginable,” etc. As the state described becomes better, and the scale is elongated, other health states are compressed to lower values. The AQoL-6D “best health” scenario is exceptionally good (“all jobs done quickly/efficiently; running very easy; very happy; never worried/sad; always full of energy; completely cope with life's problems, etc.”). This depresses numerical values of other health states.

Conclusion

  1. Top of page
  2. ABSTRACT
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. References

The literature suggests that different MA instruments produce very different results [11]. This does not indicate that CUA is an inappropriate methodology for evaluating programs. The alternative is to ignore QoL or use subjective judgments. While comparison between these options has not been reported in the literature, it is generally true that systematic approaches to problem solving outperform ad hockery. Depending upon study objectives, differences between instruments need not indicate invalidity in the measurement of QoL if the same instrument is used consistently. Invalidity will occur only if the evaluation compares the benefits of QoL with the benefits of life extension with an instrument where this implied “exchange rate” has not been validated.

CUA is evolving, and the present study employed new methods. To the authors' knowledge, this is the first time that TTO exercises have been completed by adolescents and in a classroom setting. A study by Essink-Bot et al. [27] suggested that a group setting could produce acceptable results for much less cost than face-to-face individual interviews. The classroom format was considered appropriate for the OPIC project, as adolescents are accustomed to being in classes, receiving instructions as a group, before completing work on an individual basis. A conscious decision was made not to invite junior secondary students (12–14 years) in the OPIC target group given the cognitive complexity of the task. Qualitative and quantitative results suggest that the process was highly successful.

Estimation of utilities using a three-stage procedure is also unique to the AQoL instruments with the econometric correction only used, to date, in AQoL-6D and AQoL-8D (by mid-2010). This latter innovation is particularly important in view of the gross differences between scores obtained with different instruments. It ensures that estimated utilities must be within the range of values obtained independently using the holistic (quasi-gold standard) methodology.

The utility weights in the AQoL-6D algorithm have been revised separately for adolescents in each of the four countries (Table 6). The AQoL6D can now be validly used in the economic evaluation of the OPIC interventions, and also in the field for the evaluation of any other adolescent programs in Australia, New Zealand, Fiji, and Tonga.

Table 6.  Mean analysis: linear regression results: time trade-off (TTO) on inline image
 Bt StatConstantt StatR2
Australia0.987.350.0030.070.68
New Zealand1.205.65−0.10−0.120.53
Fiji1.005.22−0.011−0.010.49
Tonga0.992.900.0030.020.23

We acknowledge the contribution made by the students in the four countries who gave their time to participate in the TTO exercises. We also express thanks to the schools, principals, and teachers who facilitated this process, and to parents who gave permission for their children to participate. Thanks are also extended to OPIC team members in each country who sought the cooperation of the schools in the process, and made logistical arrangements.

Source of financial support: National Health and Medical Research Council (Australia), Health Research Council (New Zealand), Wellcome Trust (Fiji and Tonga).

References

  1. Top of page
  2. ABSTRACT
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. References
  • 1
    Coyne T. Lifestyle Diseases in Pacific Communities. Noumea, New Caledonia: Secretariat of the Pacific Community, 2000.
  • 2
    Russell D, Parnell W, Wilson N. NZ Food, NZ People: Key Results of the 1997 National Nutrition Survey. Wellington: Ministry of Health, New Zealand, 1997.
  • 3
    Parnell W, Scragg R, Wilson N, et al. NZ Food NZ Children: Key Results of the 2002 National Nutrition Survey. Wellington: Ministry of Health, New Zealand, 2003.
  • 4
    World Health Organisation. World Health Report 2002: Reducing Risks, Promoting Healthy Life. Geneva: World Health Organisation, 2002.
  • 5
    Swinburn B, Pryor J, McCabe M, et al. The Pacific OPIC Project (Obesity Prevention in Communities)—objectives and designs. Pac Health Dialog 2007;14:13946.
  • 6
    Drummond M, O'Brien B, Stoddart G, et al. Methods for the Economic Evaluation of Health Care Programmes (2nd ed.). Oxford: Oxford University Press, 1997.
  • 7
    Keeney R, Raiffa H. Decisions with Multiple Objectives: Preferences and Value Tradeoffs. New York: Wiley, 1976.
  • 8
    Torrance G, Boyle M, Horwood S. Application of multi-attribute utility theory to measure social preferences for health states. Oper Res 1982;30:104369.
  • 9
    Torrance G. Measurement of health state utilities for economic appraisal. J Health Econ 1986;5:130.
  • 10
    Hawthorne G, Richardson J, Osborne R. The Assessment of Quality of Life (AQoL) instrument: a psychometric measure of health-related quality of life. Qual Life Res 1999;8:20924.
  • 11
    Hawthorne G, Richardson J, Day N. A comparison of the Assessment of Quality of Life (AQoL) with four other generic utility instruments. Ann Med 2001;33:35870.
  • 12
    Richardson J, Day N, Peacock S, et al. Measurement of the quality of life for economic evaluation and the Assessment of Quality of Life (AQoL) Mark 2 instrument. Aust Econ Hist Rev 2004;37:6288.
  • 13
    James K, Foster S. Weighing up disability (comment). Lancet 1999;354:878.
  • 14
    Jelsma J, Chivaura V. The global burden of disease disability weights (comment). Lancet 2000;355:207980.
  • 15
    Badia X, Roset M, Herdman M, et al. A comparison of United Kingdom and Spanish general propulation time trade-off values for EQ-5D health states. Med Decis Making 2001;21:716.
  • 16
    Tsuchiya A, Ikeda S, Ikegami N, et al. Estimating an EQ-5D population set: the case of Japan. Health Econ 2002;11:34153.
  • 17
    Jelsma K, Hansen K, De Weerdt W, et al. How do Zimbabweans value health states? Popul Health Metr 2003;1:110.
  • 18
    Johnson J, Luo N, Shaw J, et al. Valuations of EQ-5D health states: are the United States and United Kingdom different? Med Care 2005;43:2218.
  • 19
    Iezzi A, Richardson J. Measuring Quality of Life at the CHE: Databases, Their Use and Availability, Research Paper 42. Melbourne: Centre for Health Economics, Monash University, 2009.
  • 20
    Tilling C, Devlin N, Tsuchiya A, Buckingham K. Protocols for time trade-off valuations of health states worse than dead: a literature review. Med Decis Making. Online First, 12 January 2010, DOI 0272989X09357475.
  • 21
    Richardson J, Hawthorne G, Day N, et al. Difficulty with Life and Death: Methodological Issues and Results from the Utility Scaling of the Assessment of Quality of Life (AQoL) Instrument, Working Paper 1998 (70). Melbourne: Centre for Health Program Evaluation, 1998.
  • 22
    Von Winterfeldt D, Edwards W. Decision Analysis and Behavioral Research. Cambridge: Cambridge University Press, 1986.
  • 23
    Feeny D. The multi attribute utility approach to assessing health related quality of life. In: JonesA, ed. The Elgar Companion to Health Economics. Northampton, MA: Edward Elgar, 2006.
  • 24
    Prosser LA, Hammitt JK, Keren R. Measuring health preferences for use in cost–utility and cost–benefit analyses of interventions in children. Pharmacoeconomics 2007;25:71326.
  • 25
    Keren R, Pati S, Feudtner C. The generation gap. Differences between children and adults pertinent to economic evaluations of health interventions. Pharmacoeconomics 2004;22:7181.
  • 26
    Solans M, Pane S, Estrada M-D, et al. Health-related quality of life measurements in children and adolescents: a systematic review of generic and disease-specific instruments. Value Health 2008;11:74264.
  • 27
    Essink-Bot M, Perenira J, Packer C, et al. Cross-national comparability of burden of disease estimates: the European disability weights project. Bull World Health Organ 2002;80:64452.