Evaluation of an Internet-Based Disease Trajectory Decision Tool for Prostate Cancer Screening

Authors

  • Vibha Bhatnagar MD, MPH,

    Corresponding author
    1. Department of Family and Preventive Medicine, School of Medicine, University of California, San Diego, San Diego, CA, USA;
    2. Health Services Research and Development, Department for Veterans Affairs, San Diego, CA, USA;
    Search for more papers by this author
  • Dominick L. Frosch PhD,

    1. Department of Medicine, Division of General Internal Medicine and Health Services Research, University of California Los Angeles, Los Angeles, CA, USA;
    Search for more papers by this author
  • Steven R. Tally PhD,

    1. Department of Family and Preventive Medicine, School of Medicine, University of California, San Diego, San Diego, CA, USA;
    Search for more papers by this author
  • Charles J. Hamori MD,

    1. Department of Preventive Medicine, Kaiser Permanente, San Diego, CA, USA;
    Search for more papers by this author
  • Leslie Lenert MD, MS,

    1. National Center for Health Informatics, Center for Disease Control, Atlanta, GA, USA;
    Search for more papers by this author
  • Robert M. Kaplan PhD

    1. Department of Health Services, School of Public Health, University of California, Los Angeles, Los Angeles, CA, USA
    Search for more papers by this author

  • [Correction added after online publication 12-Aug-2008: Affiliations for authors have been updated]

Vibha Bhatnagar, University of California San Diego, VA Medical Center San Diego, 3350 La Jolla Village Drive, 111N-1, San Diego, CA 92131, USA. E-mail: vbhatnag@ucsd.edu; Vibha.bhatnagar@va.gov

ABSTRACT

Objective:  To evaluate the application of a chronic disease model (CDM) for prostate cancer to visual analog scale (VAS) and time trade-off (TTO) decision tools.

Methods:  A total of 138 men (mean age 58 years) viewed a CDM module for prostate cancer with and without prostate specific antigen (PSA) screening. Participants rated their hypothetical quality of life with potential prostate cancer treatment complications using a CDM-based VAS decision tool. They were then asked to estimate how many years they would be willing to trade to be free of treatment complications using a CDM-based TTO decision tool. The consistency between VAS and TTO scores and the relationship between scores and preferences for PSA screening test and hypothetical treatment choice for prostate cancer were then evaluated.

Results:  There was a significant relationship between the VAS and TTO ratings (regression P < 0.001). The TTO tool was sensitive to age. Mean scores with standard deviations for those less than 58 years compared to those 58 years and more were 7.78 (1.75) and 8.41 (1.52), respectively (P = 0.04). Using the VAS tool, men who chose PSA screening had higher quality of life ratings compared to men who did not choose PSA screening: 7.73 (1.78) and 6.59 (2.39), respectively (P = 0.01). Similar results were found with the TTO decision tool: 8.33 (1.45) and 7.04 (2.00), respectively (P = 0.005). Men who would hypothetically prefer treatment for moderately differentiated prostate cancer also had higher TTO scores compared to men who preferred watchful waiting: 8.54 (1.39) and 7.85 (1.73), respectively (P = 0.04).

Conclusion:  CDM-based for prostate cancer, VAS and TTO ratings were consistent and were concordant with patient preferences for screening; TTO ratings were also concordant with treatment choice. The use of the CDM-based TTO ratings to adjust for quality of life in decision analytic modeling needs to be explored.

Introduction

Many medical problems present the physician and patient with multiple therapeutic options and no clear best choice. Patient's values and preferences are important in the decision-making process because they can vary considerably. Decision aids are intended to facilitate decisions that reflect a high level of disease-specific knowledge, and that reflect the patient's values and preferences [1–4].

For example, visual analog scale (VAS) ratings and utility weights are often used to measure patient preference. Utility weights are also quantitative preference measures used to adjust for quality of life in decision analytic modeling. The elicitation of a utility weight, however, is not straightforward because it involves a trade-off, either in life-years [time trade-off (TTO)] or risk of death (standard gamble). Moreover, these ratings are done for a single health state and are assumed to be independent of a disease or prognosis. As a result, these exercises are often perceived as contrived and only abstractly related to actual choices. These exercises are also cognitively burdensome, and recent research has raised concerns about the consistency between utility elicitation methods [5]. Because of these issues, we have developed a new method based on a chronic disease model (CDM) to visually display how quality of life changes over a life course. Alternative life courses can be depicted to illustrate the impact of different choices on quality of life.

In this initial study, we applied this approach to prostate specific antigen (PSA) screening decisions. After viewing CDM disease trajectories with and without PSA screening, men then rated their perceived quality of life with potential treatment complications resulting from either surgery or radiation for moderately differentiated prostate cancer, the most common form of prostate cancer diagnosed after PSA screening, using a CDM-based VAS decision tool [6]. They were then asked how many years they were willing to trade, in 0.5-year increments, to be free of treatment complications using a CDM-based TTO decision tool. The TTO also used a time line to visually illustrate the trade-off patients have to make in the event of receiving a diagnosis of moderately differentiated prostate cancer: a relatively normal life course with a small risk of an earlier death from prostate cancer versus a longer life course with potential lifelong prostate cancer treatment complications [6–8].

The purpose of this study was to evaluate whether VAS and TTO decision tools based on a CDM for prostate cancer could facilitate quality decision-making. First, we compared ratings between VAS and TTO methods, and explored the relationship between VAS and TTO ratings and baseline demographics such as age and marital status. We then determined whether these tools could result in quality decision-making based on prostate cancer knowledge, as well as consistency with patient preferences for PSA screening and treatment (surgery or radiation), if they were to hypothetically have diagnosis of prostate cancer.

Methods

Study Design and Overview

This study was part of a randomized trial comparing Internet-based decision aids for PSA screening, details of which are presented elsewhere [9]. Briefly, men aged 50 years or older who were scheduled for a screening medical exam were invited to participate. Volunteers were randomly assigned to one of four intervention arms: 1) a traditional decision aid; 2) CDM for prostate cancer (the CDM intervention included VAS and TTO exercises); 3) a combination arm that included an abbreviated traditional decision aid and CDM, presented in random order; and 4) a control arm that asked participants to view prostate cancer Web sites. In this study, we focused on participants who completed VAS and TTO ratings from the CDM and combination arms. Nevertheless, because the traditional decision aid could influence VAS and TTO ratings in the combination arm, only participants who completed the CDM first are included here; only 4 of 77 of these participants continued on to view the traditional decision aid and thus, most of these men were not likely to have been influenced by the traditional decision aid.

The introduction to the CDM provided an overview of issues to consider before PSA screening, followed by animated time lines or disease trajectories. Participants were then asked to complete VAS and TTO exercises. Reviewing the complete intervention required about 30 minutes. Participants then completed a follow-up questionnaire. This included a validated 10-item prostate cancer knowledge questionnaire [10], as well as questions on whether they chose PSA testing and whether they would choose treatment (surgery or radiation) if they were to hypothetically have diagnosis of moderately differentiated prostate cancer. Because we were interested in patient preference for PSA testing, this study focused on self-reported screening choices.

CDM Development

The CDM was animated using IMPACT 4, an XML and Flash-based software that constructed preference surveys, decision support tools, and behavioral change tools [11]. Our CDM for prostate cancer was developed based on a review of the literature, and included information on prostate cancer, PSA screening (predictive value, false positives and negatives [12–19], and treatment options for clinically localized prostate cancer (complications and prognosis with and without active treatment [7,20–51]). During model development, the content was modified and validated based on feedback from a group of health professionals, all of whom were nurses or doctors actively involved in the management of prostate cancer patients. Pilot volunteers then viewed the CDM and were allowed to make comments and ask questions as they proceeded through the site. After viewing the site, they were asked questions regarding the amount of information presented, clarity of the information, length of the presentation, and balance of information. Information from these pilot tests was used to refine the CDM ultimately used in the study.

Applying the CDM

Before viewing the prostate cancer CDM, the intervention introduced users to the concept of a disease trajectory by using a general example to visually illustrate how quality of life can change over time, finally ending with death. The trajectory depicted a life course as a line-graph, with quality of life on the y-axis and time on the x-axis. Quality of life (on the y-axis) was anchored between excellent or best health and death, as described in the training exercises and life course descriptions. The action-flow added elements to the display as the accompanying narrative soundtrack described likely events from the ages of 55 to 75 years (the approximate median age of death for American men) for two hypothetical life courses.

The first trajectory described a life course for someone with moderately differentiated prostate cancer who eventually died without having had PSA screening or a diagnosis of prostate cancer; death was most likely from coronary disease. This scenario, however, also included the small possibility of a shorter life course as a result of dying from prostate cancer [21,52]. The other trajectory described someone who was screened, received diagnosis of, and actively treated for prostate cancer. This life course also included the risks for and descriptions of potential treatment-related complications [20,22]. Death was assumed to be from other causes (most likely coronary disease) and not from prostate cancer.

CDM-Based VAS and TTO Decision Tools

After viewing the prostate cancer CDM with and without PSA screening, participants then completed CDM-based VAS and TTO decision tools. Before completing the VAS and TTO decision tools, participants were given practice examples for both tools using “wearing glasses” and “blindness” as hypothetical health states. They were then asked how they would rate their overall quality of life with potential long-term prostate cancer treatment complications using a VAS decision tool. The VAS was anchored between 0 (death) and 100 (excellent or best health), and increased in 10-unit increments. The VAS score was divided by 10, resulting in a score with a range similar range to the TTO score (0 to 10), for analyses.

This was followed by a TTO decision tool that asked participants how many years they would be willing to trade to be free of prostate cancer treatment symptoms. The TTO tool used a visual display similar to the CDM. As noted above, the y-axis represented quality of life anchored between excellent (or best) health and death. The time line illustrated how changes in health affect quality of life, relative to baseline health. The TTO tool was based on a 10-year trajectory and measured how many years participants would be willing to trade, in 0.5-year increments, to be free of prostate cancer treatment-related side effects (Fig. 1).

Figure 1.

Time trade-off (TTO) decision tool. The TTO tool animated a hypothetical life course with and without prostate cancer treatment. Based on a 10-year trajectory, participants were asked how many years they would be willing to trade, in 0.5-year increments, to be free of prostate cancer treatment related side effects. For example, if they would prefer 8.5 but not 8.0 years without treatment complications, 10 years with treatment complications would be equivalent to around 8.25 years without treatment complications.

Participants were first asked whether they would prefer a full 10-year life course with potential treatment complications or a shortened life course (9.5 years) without treatment complications. If they chose a full life course, this was interpreted to mean that they were not willing to trade any significant time to be free of treatment-related side effects; if they chose the shortened life course without treatment complications, they were willing to trade at least 0.5 year to be free of treatment-related side effects. The shorter life course without treatment complications was then decreased by 0.5 year increments until the participant was not willing to trade any more time to be free of complications. For example, if they preferred 8.5 years (but not 8.0 years) without treatment complications, 10 years with treatment complications would be equivalent to around 8.25 years (the midpoint between 8.0 and 8.5) without treatment complications.

Analyses

Baseline demographics were compared between the two groups that completed the CDM. The VAS and TTO scores were then reviewed to identify discordant (illogical) ratings and to calculate failure rates for the VAS and TTO tools. Scores that were low on one scale (less than 3) but high on the other (more than 7) were considered discordant; because it was not clear whether the participant understood the exercises, these were not analyzed further. Failure for a scale was defined as scoring low on that scale but high on the other (as defined above). Typical logic checks, for example, expected ranking of scores for one versus two symptoms, were not included in this study because only one hypothetical life trajectory was measured. Also, because there was only one life course and a fixed order of presentation of VAS followed by TTO, procedural invariance was not measured.

After removal of discordant scores, the relationship between VAS and TTO scores was explored using least-squared regression; similarly, the relationship between scores and baseline demographics was then explored. Prostate cancer knowledge scores were also determined; these results are only briefly described in this article as they have been previously detailed [9]. Differences in mean VAS and TTO scores by PSA screening choice, and hypothetical treatment choices were first explored using analysis of variance (ANOVA). Adjustment for age and other demographic factors were then explored using logistic regression. All analyses were done using STATA 9.2. (Stata Corp LP, College Station, TX)

Results

Demographics and Questionnaire

Two-hundred thirty men were assigned to view the CDM decision tool, 153 were from the CDM only intervention and 77 from the combination intervention (only those who completed the CDM first are included here, as noted above). Of these, 133 (58%) completed the VAS and TTO exercise. The mean age was comparable between the two groups: 58 (+/− standard deviation 6) and 59 (5) years, respectively (Table 1). The majority of men were married (or living as married), college graduates, and white (Table 1). Among those who answered the postintervention questionnaire, the majority chose PSA screening in both groups (83% and 85%, respectively; Table 1). A minority of participants, however, would hypothetically choose treatment (surgery or radiation) over watchful waiting if they were to receive diagnosis of moderately differentiated prostate cancer (29% and 21%; Table 1).

Table 1.  Baseline demographics and preliminary outcomes by randomization group
 CDM only (n = 153)Combination, completed CDM first (n = 77)
  • *

    Among those who answered question.

  • Among those who completed VAS or TTO exercise.

  • Knowledge score in the Internet only group was 7.49 (2.34); P = 0.03 CDM groups combined compared to Internet group.

  • CDM, chronic disease model; SD, standard deviation; VAS, visual analog scale; TTO, time trade-off.

Age (+/− SD)58 (+/−6)59 (+/−5)
Married or living as married (%)117 (76)58 (75)
College graduate (%)107 (70)56 (72)
White (%)127 (83)69 (89)
Chose to have PSA screening (%)*104 (83)56 (85)
Would choose treatment (surgery or radiation) if received diagnosis of moderately differentiated prostate cancer (%)*35 (29)30 (21)
Mean VAS Score (+/− SD)7.6 (+/−2.0)7.3 (+/−2.3)
Mean TTO Score (+/− SD)7.8 (+/−2.3)7.8 (+/−1.6)
Knowledge Score (+/− SD)7.96 (+/−2.03)7.95 (+/−2.30)

CDM-Based VAS and TTO Scores and Prostate Cancer Knowledge

The mean VAS scores were similar between the two groups: 7.6 (2.0) and 7.3 (2.3), CDM only and combination, respectively. Mean TTO scores were also similar: 8.2 (1.7) and 7.8 (1.6) (Table 1). Both groups had comparable prostate cancer knowledge scores; as detailed elsewhere [9], knowledge scores for these groups were higher than the scores for participants randomized to view public Internet Web sites focused on PSA screening (significantly higher with CDM groups combined, P = 0.03). Because there were no significant baseline differences, the two groups were then merged.

CDM-Based VAS and TTO: Discordant Results, Failure Rates and Consistency

Only 2 of the 133 participants with low VAS scores (2.0 and 2.0) had discordant (illogical) TTO scores (8 and 9.5), an approximate 2% failure rate for the VAS tool. Five participants had low TTO scores (0 to 3) with high VAS scores (8.4 to 10.0), an approximate 5% failure rate. With the exclusion of these 7 outliers (an overall 5% failure rate), 126 participants were available for further VAS and TTO analyses. There was a significant relationship between VAS and TTO scores (R = 0.2, P < 0.001; Fig. 2). Adjustment for age, marital status, education, and ethnicity did not significantly alter this relationship.

Figure 2.

Fitted chronic disease model (CDM) based time trade-off (TTO) and visual analog scale (VAS) ratings. Using least-squared regression, there was a significant relationship between CDM-based VAS and TTO ratings (R = 0.2, P < 0.001). CI, confidence interval.

CDM-Based VAS and TTO: Baseline Demographics

TTO scores, but not VAS scores, were related to age; men under the median age of 58 years had a perceived lower quality of life (were willing to trade more years to live without prostate cancer treatment complications) than those 58 years and over: 7.78 (1.75) versus 8.41 (1.52), P = 0.04 (results not shown). Adjustments using least-squared regression for marital status, education and ethnicity were not significant.

CDM VAS and TTO: PSA Screening Choice

The relationship between mean VAS scores and the decision to choose PSA screening was also significant. Men who chose PSA screening perceived a higher quality of life with potential prostate cancer treatment complications in comparison to those who did not choose PSA screening. Mean VAS scores among those who did and did not choose PSA screening were 7.73 (1.78) and 6.59 (2.39), respectively (ANOVA, P = 0.01; left portion of Fig. 3). The relationship between TTO scores and the decision to have PSA testing was even more significant: 8.33 (1.45) and 7.04 (2.00), respectively (P = 0.005; right portion of Fig. 3). Adjustments for age, marital status, education, and ethnicity using least-squared regression were not significant.

Figure 3.

Chronic disease model visual analog scale (VAS) and time trade-off (TTO) scores by prostate specific antigen (PSA) screening choice. Ratings with long-term prostate cancer treatment complications were higher among those who chose PSA screening compared to those who did not chose screening. Mean VAS scores among those who did and did not choose screening were: 7.73 (1.78) and 6.59 (2.39; P = 0.01). Mean TTO scores were: 8.33 (1.45) and 7.04 (2.00; P = 0.005).

CDM VAS and TTO: Hypothetical Treatment Choice

Men who hypothetically chose treatment for moderately differentiated prostate cancer also rated their quality of life with treatment complications higher than those who chose watchful waiting. Differences between mean scores, however, were significant for TTO only: 8.54 (1.39) and 7.85 (1.73), (P = 0.04; Fig. 4). Adjustment for age (less than 58 years or 58 years and over) using a logistic regression model was significant (P = 0.02 for age coefficient); the interaction between treatment choice and age was not significant.

Figure 4.

Chronic disease model visual analog scale (VAS) and time trade-off (TTO) scores by treatment choice. TTO ratings with long-term prostate cancer treatment complications were significantly higher among those who would hypothetically prefer treatment (surgery or radiation) for moderately differentiated prostate cancer compared to those who would not prefer treatment. Mean TTO scores were 8.54 (1.39) and 7.85 (1.73; P = 0.04).

Discussion

We developed and evaluated a novel CDM for prostate cancer that animated expected life trajectories resulting from PSA screening decisions. We then asked participants to rate their quality of life with potential prostate cancer treatment complications over an entire life course using a VAS. This was then followed by a TTO exercise that estimated how many life-years men would be willing to trade to be free of potential prostate cancer treatment complications. CDM-based VAS and TTO scores were consistent (reliable between methods). Participants also had a good fund of knowledge related to prostate cancer and PSA screening, and the intervention tools may have helped to facilitate decisions that were concordant with patient values.

The evaluation of concordance with patient values, however, was based on an assumption that men with a higher tolerance for treatment complications would be more likely to choose screening for early prostate cancer diagnosis and subsequent intervention. On the other hand, men who are less tolerant would be less likely to want PSA screening for early diagnosis and less willing to undergo treatment. While the majority of participants chose PSA screening, screening choice was consistent with VAS and TTO ratings for treatment complications for moderately differentiated prostate cancer. Based on our study, those who chose PSA screening had a higher perceived quality with potential prostate cancer treatment complications compared to those who did not choose screening using both the VAS and TTO scales. Between the two methods, TTO scores were more strongly correlated to PSA screening preference than VAS scores.

TTO scores were also related to treatment choice. The majority of study participants preferred no intervention (watchful waiting) over surgery or radiation if they were to hypothetically have diagnosis of moderately differentiated localized prostate cancer. This is strikingly different from regional US registries where only a minority of men (8% to 18.5% for all grades of localized cancer) choose expectant management [53,54], suggesting that the utilization of radiation or surgery for moderately differentiated prostate cancer that poses a limited threat to longevity may be driven by poor quality decisions. The animated display of contrasting a full life course with treatment complications or a relatively normal life with small risk of a shortened life expectancy as a result of death from prostate cancer may have been an effective way to communicate the actual trade-off in the event of a diagnosis of localized moderately differentiated prostate cancer. Nevertheless, whether findings would differ in the event of an actual diagnosis of prostate cancer and in conjunction with oncology and radiation consultations needs to be further explored.

Interestingly, the TTO, but not the VAS decision tool, was also independently related to age. Younger men were willing to trade more years to be free of prostate cancer treatment complications. Younger men may have assumed that they have more years to live, and may have placed less value additional time than older men. Younger men would have had to live longer with these complications and may have perceived the longer life course with treatment complications as more burdensome than older men. Older men may have also had experience with prostate related lower urinary obstructive urinary symptoms and erectile dysfunction; thus, they may have perceived prostate cancer treatment symptoms as less bothersome compared to men who have not had experience with these symptoms [22].

Measuring Patient Preference

As this is a novel application of CDM to measure patient preference, it is difficult to directly compare our results to other studies. Nevertheless, it is important to note that conventional VAS and TTO ratings are often inconsistent [55], raising concerns about the use of such scales in clinical practice. Conventional TTO utility exercises are framed under conditions of certainty and are not presented in the context of a disease or prognosis. Conventional approaches also measure utility weights for single-item health states; this is unrealistic for many chronic medical conditions such as long-term side effects resulting from prostate cancer treatment [22]. Based on our previous study, we noted that quality of life diminished with increasing number of complications. Aside from directly measuring utility weights for combination health states, there was no clear way to derive utility weights for combination health states based on single-item utility weights [56].

The use of contrasting life trajectories may have been an effective way to measure preferences for multi-symptom health states in the context of a diagnosis and prognosis over many years. The outcomes were also framed under conditions of uncertainty, a more accurate representation of circumstances under which patients make most treatment decisions for complicated medical illnesses. In this sense, our approach is similar to the healthy life-year equivalent model proposed by Mehrez, Birch and Gafni [57,58]. The effects of individual health states, however, cannot be determined with this approach, and changes in technology (subsequent complication rates etc.) would necessitate a revision of the life course description. Nonetheless, effective risk communication is extremely important because patients need to understand the limitations, benefits, and expected outcomes of medical interventions. In the case of PSA screening, decision aids must effectively communicate complex information on the limitations, uncertain benefits, and future consequences of PSA screening.

Implications for Decision Modeling

Finally, utility weights derived from conventional TTO exercises are used in decision analytic models to adjust for quality of life. Traditional models, however, incorporate outcomes for individual and combination health states separately, requiring corresponding individual and combination utility weights. Using prostate cancer treatment outcomes as an example, utilities for any combination of sexual, bowel, urinary need to be measured [21,56]. Direct measurement of all these health states is a burdensome task, often resulting in unreliable weights [55]. Alternatively, a model could hypothetically be built based on expected life trajectories. By assigning a single weight to a life course, the decision tree can be pared down to one of several arms with corresponding adjustment for quality of life. How a simplified model compares to a more traditional model remains to be determined.

Study Limitations

Because this is an exploratory study, there are several important limitations. Although our study was Internet-based and limited to those with a cable or digital subscriber line modem, the software could be developed to run on any computer platform. Results from the TTO only and combination arms from the original intervention study were analyzed together because baseline parameters and initial results were similar between these groups. Study participants were mainly white, highly educated, employed, and computer literate. We do not know how acceptable this method would be to less-educated and more diverse populations.

Low completion rates for the VAS and TTO exercises also need to be acknowledged. Although there may be several reasons (viewer fatigue, cognitive burden, etc.), future studies need to include prompts to encourage completion of the decision tools. Because we explored only one life course, logic checks were also not built into the study. The tools were also placed in a fixed order, VAS followed to by the TTO, so that participants rated perceived quality of life using an easier scale before proceeding to a more complicated TTO scale. This fixed order may have contributed to the good concordance between the two scales; how the VAS exercise influenced the TTO results, and whether the TTO tool could be used on its own, cannot be determined from this study. Similarly, because all participants viewed the CDM for prostate cancer and then completed the VAS and TTO decision tools, we cannot determine whether the CDM alone would result in similar decisions for PSA screening and hypothetical choices for prostate cancer treatment.

Therefore, future studies should ideally explore a CDM module with and without decision tools. Although we explored actual PSA screening decisions, treatment decisions were hypothetical and future work needs to explore the CDM-based VAS and TTO decision tools in the event at actual treatment decisions. TTO scores also appear to be age-sensitive but because of our narrow age range, the psychometric properties with respect to age could not be fully explored here.

Conclusions

In summary, we have developed a new approach to informed decision-making by using a CDM and building upon VAS and TTO methodology. This study provides preliminary evidence suggesting that VAS and TTO ratings based on a CDM for prostate cancer in middle-aged and older men are reliable methods for measuring quality of life (concordant ratings between methods with overall low failure rates). TTO ratings may be superior to VAS ratings because they were more sensitive to PSA screening choice and were also related to hypothetical treatment choice. This application of CDM methodology is a promising approach to facilitate high-quality decision-making.

Source of financial support: CDC U57/CCU920678-04-1, Department for Veterans Affairs.

Ancillary