ABSTRACT
 Top of page
 ABSTRACT
 Introduction
 Background
 Methods
 Results
 Discussion
 Conclusion
 References
Objectives: Losses to followup and administrative censoring can cloud the interpretation of trialbased economic evaluations. A number of investigators have examined the impact of different levels of adjustment for censoring, including nonadjustment, adjustment of effects only, and adjustment for both costs and effects. Nevertheless, there is a lack of research on the impact of censoring on decisionmaking. The objective of this study was to estimate the impact of adjustment for censoring on the interpretation of costeffectiveness results and expected value of perfect information (EVPI), using a trialbased analysis that compared rate and rhythmcontrol treatments for persons with atrial fibrillation.
Methods: Three different levels of adjustment for censoring were examined: no censoring of cost and effects, censoring of effects only, and censoring of both costs and effects. In each case, bootstrapping was used to estimate the uncertainty incosts and effects, and the EVPI was calculated to determine the potential worth of further research.
Results: Censoring did not impact the adoption decision. Nevertheless, this was not the case for the decision uncertainty or the EVPI. For a threshold of $50,000 per lifeyear, the EVPI varied between $626,000 (partial censoring) to $117 million (full censoring) for the eligible US population.
Conclusions: The level of adjustment for censoring in trialbased costeffectiveness analyses can impact on the decisions to fund a new technology and to devote resources for further research. Only when censoring is taken into account for both costs and effects are these decisions appropriately addressed.
Introduction
 Top of page
 ABSTRACT
 Introduction
 Background
 Methods
 Results
 Discussion
 Conclusion
 References
One of the major challenges faced by decisionmakers in all (budgetconstrained) healthcare systems is the choice between alternative interventions for the same medical indication. Increasingly these decisions are being guided by economic evidence, including results of costeffectiveness studies. Inevitably, the estimates of the costs and effects involve some uncertainty because of measurement, sampling, and random errors. This leads to a situation in which decisionmakers must address two decisions: the first involves identifying the most appropriate method of patient management to fund given the current level of information and uncertainty, and the second involves funding additional research to reduce the uncertainty in the future [1]. A formal framework exists to address these two separate but related decisions [1–3]. Given the objective to maximize health benefit subject to a budget constraint, the appropriate method of patient management is identified, within this framework, according to the expected costeffectiveness of the interventions (i.e., the point estimate), irrespective of the uncertainty surrounding the estimate [1]. The second decision, that of whether to fund more research, involves an assessment and valuation of the uncertainty surrounding the decision [1].
Prospective collection of patientlevel cost data within randomized controlled trials is one approach to obtain the information needed to estimate costeffectiveness. In the majority of trials, however, these data are incomplete as a result of censoring and this needs to be accounted for in the costeffectiveness analysis [4]. In this situation, the estimation of the expected costs and effects from the sample becomes more involved with implications for the costeffectiveness analysis. Investigators have examined the impact of different levels of adjustment for censoring, ranging from nonadjustment, through adjustment of effects alone or costs alone, to adjustment for both costs and effects (see Young [4] for a comprehensive review and description of approaches for accounting for censored costs). Nevertheless, there is a lack of research on the impact of these adjustments for censoring on the measure of uncertainty and the decision to fund more research.
This article examines the impact of not adjusting for censoring versus adjusting for censoring of effects alone and adjusting for censoring of both effects and costs, on healthcare decisionmaking in terms of the optimal intervention to adopt and whether to fund the collection of additional information. This is demonstrated using patientlevel data from a trialbased costeffectiveness analysis comparing ratecontrol to rhythmcontrol treatment for atrial fibrillation (AFFIRM) [5].
Results
 Top of page
 ABSTRACT
 Introduction
 Background
 Methods
 Results
 Discussion
 Conclusion
 References
Figure 1 presents the Kaplan–Meier curves illustrating the survival over time for the scenario when survival is not adjusted (no adjustment) and the scenarios when survival is adjusted (partial adjustment and full adjustment).
Table 1 reports the mean survival, total cost, and ICER associated with rate versus rhythmcontrol treatment for the three scenarios that differed in how adjustments were made for censoring. These are all computed as sample averages.
Table 1. Mean expected survival, total cost, and incremental costeffectiveness ratio (ICER) for each censoring scenario Censoring  Total cost ($)  Mean survival (years)  Incremental (rate—rhythm)  ICER of rate vs. rhythm ($ per lifeyear)  Decision 

Rate  Rhythm  Rate  Rhythm  Cost ($)  Survival (years) 


None  20,595  25,375  3.1869  3.1878  −4,800 (−6,624 to −2,923)  −0.0009 (−0.08 to 0.07)  4,983,477  Chose ratecontrol 
Partial*^{†}  20,595  25,375  4.6749  4.5983  −4,800 (−6,624 to −2,923)  0.08 (−0.01 to 0.17)  Ratecontrol dominates  Chose ratecontrol 
Full^{‡}  32,048  35,509  4.6749  4.5983  −3,461 (−8,438 to 1,810)  0.08 (−0.01 to 0.17)  Ratecontrol dominates  Chose ratecontrol 
In each case, ratecontrol is less costly on average than rhythmcontrol. In the two scenarios where censoring of survival is taken into account, ratecontrol is also more effective on average and dominates rhythmcontrol. When no account is made for censoring, ratecontrol is associated with a nonsignificant shorter mean survival (−0.0009 years, 95% CI −0.08 to 0.07) compared to rhythmcontrol. Nonetheless, after considering the joint distribution of costs and effects, ratecontrol remains the favored approach because of the lower cost compared to rhythmcontrol (–$4800, 95% CI –$6624 to –$2923), despite the absence of a statistically significant difference in survival between the two strategies. Therefore, in each case, the decision made on the basis of expected ICER estimates is to adopt ratecontrol, either because it dominates (where censoring is applied) or because the associated ICER (saving almost $5 million for every lifeyear given up in the case of no censoring) exceeds any reasonable threshold. When both the incremental cost and incremental survival are negative, the decision rule becomes select if ICER greater than threshold.
Figure 2 illustrates the incremental costeffectiveness plane, for the comparison between ratecontrol and rhythmcontrol, for each scenario. Each point represents one replicate (incremental cost and incremental survival) from the bootstrap. Figure 2a represents the case where censoring is not taken into account. The location and spread of the incremental costeffect pairs in the vertical direction indicates that there is no uncertainty regarding the existence of costsavings with the ratecontrol strategy compared to the rhythmcontrol strategy (all points fall below the horizontal axis), although there is some uncertainty about the magnitude of the costsavings (incremental savings vary from $2923 to $6624). With regard to effectiveness, there is uncertainty regarding whether and the extent to which ratecontrol confers a survival benefit compared to rhythmcontrol (from −0.08 to 0.07 years). This is consistent with the finding of a nonsignificant difference in survival gain between the two treatment groups. Approximately onehalf of replicates (51.1%) were located to the left of the vertical axis (negative incremental survival), indicating that there was considerable uncertainty surrounding the effectiveness of ratecontrol.
Figure 2b represents the case where censoring is taken into account in terms of survival only. The location of the replicates in the vertical (cost) plane is the same as the scenario with no account for censoring. Within the horizontal plane, the location of the replicates indicates that there is much less uncertainty about whether ratecontrol is effective compared to rhythmcontrol (now only 4.6% of replicates involved negative incremental survival). Nevertheless, the spread of the replicates indicates that there is slightly more uncertainty surrounding the extent of the survival difference between the two treatments (−0.01 to 0.17 years).
Figure 2c represents the case where censoring is taken into account in terms of both costs and survival. The location and spread of the incremental costeffect pairs within the vertical plane indicates that there is uncertainty regarding the existence and extent of costsavings with the ratecontrol strategy in comparison to the rhythmcontrol strategy (+$1810 to –$8438). The majority (90.4%) of the replicates were located below the horizontal axis (negative incremental cost), indicating that ratecontrol was most often costsaving compared to rhythmcontrol. Within the horizontal plane, the replicates were identical to censoring of survival only (Fig. 2b). In addition, there was a small proportion of replicates (0.48%) that were located both above the horizontal axis (positive incremental cost) and to the left of the vertical axis (negative incremental survival), indicating the potential for ratecontrol to be both more costly and less effective than (dominated by) rhythmcontrol.
Figure 3 illustrates the costeffectiveness acceptability curves for ratecontrol compared to rhythmcontrol for each of the levels of censoring adjustment, calculated from the bootstrap replicates as the proportion where ratecontrol is associated with the maximum NB. Given the data, over the range specified for the threshold (λ) ($0 to $100,000), the probability that ratecontrol is costeffective compared to rhythmcontrol is high (>89%) regardless of the level of adjustment for censoring (note the discontinuation of the axis). Nevertheless, the level of adjustment for censoring has an impact upon the extent of the decision uncertainty. When censoring is taken into account for survival only (partial censoring), the probability that ratecontrol is costeffective remains above 95% irrespective of the threshold. This reflects the minimal uncertainty regarding the effectiveness of ratecontrol compared to rhythmcontrol (only 4.6% of the costeffectiveness pairs involved negative incremental survival). In each of the other cases, the probability falls below 95% over some range of the threshold (λ > $73,000 for the scenario with no account for censoring, λ < $17,000 for the scenario with censoring taken into account for both costs and survival), indicating the presence of some decision uncertainty.
In addition, the costeffectiveness acceptability curve for the fullcensoring scenario has a different shape to those for the other scenarios, an increasing rather than decreasing probability that ratecontrol is costeffective as the threshold increases. This reflects the uncertainty concerning the existence of cost savings associated with ratecontrol compared to rhythmcontrol. In this scenario, 9.6% of the costeffectiveness pairs involved positive incremental cost; as such, the probability that ratecontrol is costeffective given a threshold of zero (the decisionmaker is only interested in costsavings) is below 100% (90.4%). Nevertheless, as the value of the threshold increases, these costeffect pairs begin to look costeffective and the probability that ratecontrol is costeffective increases. See Fenwick et al. for a full discussion of the relationship between costeffectiveness pairs in the costeffectiveness plane and the shape of costeffectiveness acceptability curves [15].
Table 2 details the EVPI for the population associated with the decision, given a threshold (λ) of $25,000, $50,000, and $100,000 per lifeyear gained. Using the example of $50,000 per lifeyear gained as the threshold (λ), when no account is made of censoring, the EVPI surrounding the decision is $23 million. When censoring is taken into account in both costs and survival, the EVPI surrounding the decision is $117 million. Nevertheless, when censoring is only taken into account for survival, the EVPI surrounding the decision is only $626,000. Figure 4 illustrates the population EVPI associated with the different censoring scenarios over a range of values for the threshold. In the cases where censoring for cost is not taken into account (no censoring and partial censoring), the EVPI rises. This is because both the uncertainty surrounding the decision (error probability) and the value of the threshold (value of the consequences of an error) are rising. In the case where censoring is taken into account for both costs and survival, the EVPI falls initially as the threshold rises, because the reduction in uncertainty outweighs the increased valuation of the consequences associated with an incorrect decision. As the threshold increases beyond $53,000 per lifeyear, the EVPI rises with the threshold. This corresponds to the point where the costeffectiveness acceptability curve levels off (the reduction in the decision uncertainty slows), and thus reflects the fact that the value of the consequences of an error (measured by the threshold) outweighs the reduction in the error probability over this range.
Table 2. Expected value of perfect information (EVPI) for the eligible population for each censoring scenario at different thresholds (λ) Censoring  EVPI ($) at 

λ = $25,000  λ = $50,000  λ = $100,000 


No  11,000  23 million  831 million 
Partial*  0  626,000  33 million 
Full^{†}  179 million  117 million  159 million 
Discussion
 Top of page
 ABSTRACT
 Introduction
 Background
 Methods
 Results
 Discussion
 Conclusion
 References
In this article we examine the impact of the level of censoring adjustment undertaken on the two decisions faced by a rational decisionmaker using data from a large trial of two strategies for treating atrial fibrillation [5]. The trial showed a small, nonsignificant difference in survival between the two strategies. Only 16% of the subjects died; thus 84% were censored. This high amount of censoring is a common situation in the literature.
We found that the level of adjustment for censoring did not affect the decision about which intervention to adopt, despite the converse effectiveness result associated with the uncensored analysis (that rhythmcontrol was more effective than ratecontrol). Nevertheless, the level of censoring may impact the decision to fund additional research, because the estimated population EVPI varies by several orders of magnitude depending on the censoring scenario. For the partialcensoring scenario (survival only), EVPI is estimated as $626,000 for a threshold of $50,000 per lifeyear gained, suggesting that further research, the cost of which is likely to exceed this value, may not be worthwhile. Nevertheless, for the scenarios involving either no censoring or full censoring, the estimates of EVPI are $23 and $117 million dollars respectively, and as such, further research is likely to be worthwhile. In this case, the variations in the estimates of the EVPI across the three censoring scenarios are driven mainly by differences in the error probabilities surrounding the decisions (as measured by the inverse of the costeffectiveness acceptability curve), although differences in the consequences of making an error (as measured by the net benefits associated with the decisions) have some impact on the results. Thus, the EVPI estimates are similar for no censoring and partial censoring, over the range of thresholds for which the uncertainty associated with each is similar (up to a value of approximately $30,000 per lifeyear), and the EVPI estimate is greatest for the nocensoring scenario for values of the threshold above $65,000 per QALY (approximately), where the uncertainty surrounding the decision exceeds that associated with the other scenarios. The partialcensoring scenario is associated with the lowest level of decision uncertainty (see Fig. 3), and this is reflected in the lower EVPI estimates generated for the partialcensoring scenario.
These results should be interpreted within the context that EVPI alone is not sufficient for determining the worth of further research. In the situation where the EVPI suggests that further research would be potentially worthwhile, additional analysis could be performed to determine the EVPI for a particular parameter or group of parameters (e.g., economic parameters, clinical parameters) to assess the (potential) worth of research focused on different facets of the decision [1,16]. The process involves determining the increase in the expected value of the decision associated with resolving the uncertainty concerning a parameter or group of parameters. Nevertheless, perfect information is not achievable with a finite sample size, and the expected value of partial perfect information still provides only a maximum value for further research which can be compared to the cost to determine whether the research is potentially worthwhile (necessary condition). Determining whether specific research, with a finite sample size, is worthwhile requires an analysis of the expected value of sample information. This involves valuing the reduction in uncertainty, and hence the increase in the expected value of the decision, actually achievable through research, and depends upon the extent to which uncertainty and the associated consequences are actually reduced by the information provided from research (the informativeness of the research) [16].
There are other methods of addressing censoring in costeffectiveness analysis that we did not examine in this article. Bang and Tsiatis [27] suggested a number of nonparametricsolutions estimators of costs in the presence of censoring based upon inverse weighting techniques. Unlike those of Lin et al. [21], their estimators are shown to be consistent regardless of censoring pattern. Bang and Tsiatis [27] provide both an estimator based on total costs only (simple weighted case estimator), and an estimator that partitions the study into intervals similar to Lin et al. [21]. Willan et al. [39] provide methods to estimate the mean and variance of the incremental net benefit statistic under conditions of censoring. These methods can be applied when the health outcome is either mean survival time or mean qualityadjusted survival. More recently, the application of regression techniques to estimate costs and effects in the presence of censoring has been discussed in the literature [28,40,41]. One of the advantages of using regression techniques is the ability to include covariates in the estimation of costs and effects in the presence of censoring.
We selected the KMSA approach described by Lin et al. [21] for censoring, because this approach has been shown to provide consistent estimators of average costs if it is assumed that censoring occurs at the boundaries of the intervals. Furthermore, it is the most commonly applied approach in the literature. Nevertheless, the method does have limitations because there is no reason to expect censoring to occur at the boundaries of the selection intervals. Therefore, the assumption about the consistency of the estimators will likely be violated to some degree in most cases [21].