Cost-Effectiveness and Value of Information Analyses of Neuraminidase Inhibitors for the Treatment of Influenza

Authors


Allan J. Wailoo, Health Economics and Decision Science, ScHARR, Regent Court, 30 Regent Street, Sheffield S1 4DA, UK. E-mail: a.j.wailoo@sheffield.ac.uk

ABSTRACT

Objectives:  To assess the cost-effectiveness of alternative strategies for the treatment of suspected influenza in otherwise healthy adults and to identify future research priorities using value of information analysis.

Methods:  A decision model was used to estimate the costs and effects, in terms of quality-adjusted life-years (QALYs) of amantadine, zanamivir, and oseltamivir for the treatment of influenza in otherwise healthy adults using data predominantly from meta-analysis of randomized controlled trials. Probabilistic sensitivity analysis using Monte Carlo simulation was conducted. The expected value of perfect information for the entire model and for individual parameters was calculated.

Results:  Based on mean costs and effects, zanamivir is dominated by oseltamivir. The incremental cost-effectiveness ratio for amantadine (compared with no treatment) is £11,000 and £44,000 for oseltamivir (compared with amantadine). The probability that amantadine is cost-effective at a willingness to pay of £30,000 per QALY is 0.74, falling to 0.49 at £20,000 per QALY. Global expected value of perfect information (EVPI) is £2 m over 15 years if a willingness to pay threshold of £30,000 per QALY is assumed rising to £9.6 m at £45,000 per QALY. EVPI for only one parameter exceeds £500,0000 at £30,000 per QALY: the quality of life for untreated influenza.

Conclusions:  At traditionally accepted values of willingness to pay for health benefits, it is unlikely that additional research would be an efficient use of scarce resources. The only exception to this would be to examine the health-related quality of life impact of influenza in an untreated patient group. If a higher threshold value were acceptable, there are a small group of parameters that may warrant further investigation. These would, however, require comparative, potentially expensive, research studies.

Introduction

Influenza epidemics of varying intensity occur most winters. The condition is usually self-limiting in people who are relatively healthy, with typical symptoms such as headache, fever, sore throat, cough, and aching muscles and joints lasting several days. Nevertheless, more severe, predominantly respiratory complications, such as pneumonia and bronchitis, are the source of substantial morbidity and increased mortality associated with influenza epidemics. In England and Wales, an estimated 6200 to 29,600 additional deaths occurred during each of the epidemics between 1975 and 1976 and 1989 and 1990 [1], about 10 times the actual number of death certifications for influenza over that time, suggesting that influenza is responsible for many hidden deaths.

The principal component of public health strategies aimed at controlling the burden of influenza is vaccination. In the UK, National Health Service vaccination is offered to those considered at elevated risk of influenza complications, either because of age or because of concomitant disease, with uptake levels running at 69% between 2002 and 2003 [2]. Two adamantanes, amantadine (Lysovir; Alliance Pharmaceuticals Ltd., Chippenham, UK) and rimantadine (Flumadine; Alliance Pharmaceuticals Ltd.), have been produced since the mid-1960s for both treatment and prevention, although the latter does not have a UK license and clinical uptake of the former has been limited because of concerns over adverse events, resistance, and its limited spectrum of activity (adamantanes operate only against the replication of influenza A).

Neuraminidase inhibitors are a new class of antiviral drugs that provide additional potential strategies for the control of influenza. Zanamivir (Relenza; GlaxoSmith- Kline, Brentford, UK) for the treatment of influenza is administered by means of an inhaler and was the subject of one of the first technology appraisals undertaken bythe National Institute for Clinical Excellence (NICE) [3] in the UK. Guidance issued in 2000 recommended NHS use should be limited to “high-risk” groups. The launch of oseltamivir (Tamiflu; F. Hoffman-La Roche, Ltd., Basel, Switzerland), taken orally and licensed for both prophylaxis and treatment, prompted further review of that guidance, and this article builds on evidence that was submitted as part of that review [4].

This study examines the cost-effectiveness of making amantadine, zanamivir, or oseltamivir available as treatments for individuals in the community with suspected influenza, compared with no treatment. A decision tree model is described and analyzed probabilistically to estimate mean costs and QALYs for each of the alternative strategies and to reflect uncertainty in these outputs arising from the combined uncertainty in model inputs. Probabilistic sensitivity analysis (PSA) is increasingly recommended in guidelines for cost-effectiveness analysis [5,6] as a means of quantifying decision uncertainty to decision-makers.

Value of information analysis (VOI) has been seen as a logical next step after PSA [7] but remains little used in health technology assessment [8]. This set of methods provide a formal framework in which the value of collecting proposed new information, and thereby reducing or eliminating uncertainty in model parameters, is calculated with respect to the impact that such reductions in uncertainty have on decision uncertainty. Additional information has a value to the extent that it reduces the probability that decision-makers make a recommendation that is “incorrect,” i.e., adopt a technology that is not in fact cost-effective.

Some of the methodological issues in VOI were first described by Claxton and Ades [9] using a simplified cost-effectiveness model of zanamivir for the treatment of influenza, and the technique is receiving increasing attention within the health economics field [10,11]. This article aims to contribute to that literature by applying the technique to a substantive current decision problem: the cost-effectiveness of antiviral treatment for influenza in the UK. The analysis is restricted to persons aged between 12 and 65 years considered to be otherwise healthy.

Methods are presented in section 2. This section describes the structure of the decision model, parameter values, and characterization of uncertainty. In addition, the methods of analysis are presented. We outline here how PSA is conducted and represented. This is followed by a description of the methods for assessing VOI. Results are in section 3 and discussion in section 4.

Methods

Model Structure

A probabilistic decision tree model was developed to assess the cost-effectiveness of competing influenza treatments in terms of additional cost to the UK NHS per additional quality-adjusted life-year (QALY) gained (Fig. 1).

Figure 1.

Model schematic.

The decision model is described in four separate stages. The decision problem for the UK NHS is described at stage one, namely, which of four alternative influenza treatment strategies should be adopted: amantadine (100 mg daily); zanamivir (10 mg twice daily); oseltamivir (75 mg twice daily); or no drug treatment. Each treatment course lasts 5 days. The decision is relevant to a patient population that consists of those with influenza-like illness (ILI) of whom only a fraction will decide to consult with a family doctor (stage 2). It is assumed that treatment with any antiviral must be made within 48 hours of symptom onset, but the model allows for individuals to receive (ineffective) treatment after that time period because symptom onset may be insidious and difficult to recall accurately. Prescription of antibiotics, instead of antivirals, may be made at this initial general practitioner (GP) consultation (stage 3) where the practitioner suspects bacterial infection rather than influenza. It is considered unlikely that both antiviral and antibiotics would be prescribed at this stage. Disease progression is described at stage 4. The first distinction made here is between those individuals who have genuine influenza rather than clinically indistinguishable conditions such as bacterial infections or other viruses (e.g., respiratory syncytial virus). Near patient tests for influenza have recently become available but are little used in the UK NHS because of concerns over cost and specificity and are therefore not considered in the model. No further costs or benefits beyond those already described are calculated in the model for patients who do not have influenza. Influenza may be either strain A or strain B and the distinction is important because amantadine is effective only for the treatment of influenza A. True influenza cases may experience complications that require a revisit to the GP and that may result in pneumonia. Appropriate antiviral treatment, i.e., treatment given to influenza-positive patients within 48 hours of the onset of illness, reduces the length of influenza illness, the probability of complications requiring additional GP visits, and the probability of developing pneumonia.

The model assumes that antiviral treatment does not affect either hospitalizations or mortality. Several previous studies have extrapolated from intermediate data [12,13] to estimate these effects. Such events, however, are extremely rare in an otherwise healthy adult population and it was not considered credible that antiviral drug treatment would impact on these severe cases, an assumption that was also adopted by the NICE technology appraisal committee in their consideration of the evidence [14].

Parameter Values

Parameter values and associated probability distributions are recorded in Tables 1 and 2. These values were derived from the best available data. Primarily, meta-analyses of randomized controlled trials (RCTs) were used, but where no such sources were available, or were considered either inappropriate or insufficient, they were supplemented or replaced with alternative data, often from multiple sources [4].

Table 1.  Cost and outcome values
Parameter description*Mean
value
(95% Confidence interval)Probability distributionSourceType of data
  • *

    Parameters used only indirectly are displayed in italics.

  • The beta distribution was applied to the QALY scores. Quality-adjusted life-days (QALDs) are reported for ease of interpretation.

  • See Equation 1.

  • §

    Reduction in length of illness = 1.32 × reduction in length of fever.

  • GP, general practitioner; RCTs, randomized controlled trials.

QALDs over 21 days—no treatment15.13214.62 to 15.66Beta (α = 3065, β = 70,877)WV15670 [15], WV15671 [16],
WV 15730, M76001
Fixed effects meta-analysis of RCTs
Difference in QALDs over
21 days—oseltamivir Tx
0.3700.363 to 0.377Normal (μ = 0.37, SD 0.004)WV15670 [15], WV15671 [16],
WV 15730, M76001
Fixed effects meta-analysis of RCTs
QALDs over 21 days—amantadine Tx15.390 Derived from QALD (no treatment, oseltamivir), length of illness (no treatment, oseltamivir, amantadine) 
QALDs over 21 days—zanamivir Tx15.456 Derived from QALD (no treatment, oseltamivir), length of illness (no treatment, oseltamivir, zanamivir) 
 Mean length of illness—no  treatment7.6906.856 to 8.596Lognormal (μ = 2.04, SD 0.06)[4]Random effects meta-analysis of RCTs
 Mean reduction in length of  illness—zanamivir1.6830.812 to 2.575Normal (SD 0.44)[4]Random effects meta-analysis of RCTs
 Mean reduction in length of  illness—oseltamivir1.9190.939 to 2.918Normal (SD 0.51)[4]Random effects meta-analysis of RCTs
 Mean reduction in length of  illness—amantadine1.280 Derived from reduction
in length of fever§
[4]Meta-regression to convert fever to illness based on RCTs
 Mean reduction in length of  fever—amantadine0.970.62 to 1.31Normal (SD 0.18)[4]Random effects meta-analysis of RCTs
QALD loss amantadine
adverse events
0.95 NoneAssumption 
QALD loss pneumonia6.348 None[20]Population valuation studies
Costs
 GP visit21.38 None[28]National unit costs
 Antibiotic4.05 None[29]; Prescriptions Pricing Authority, pers. comm.National cost
 Amantadine3.38 None[29]; Prescriptions Pricing Authority, pers. comm.National cost
 Zanamivir24.98 None[29]; Prescriptions Pricing Authority, pers. comm.National cost
 Oseltamivir19.16 None[29]; Prescriptions Pricing Authority, pers. comm.National cost
Table 2.  Probability values
Parameter descriptionMean value(95% Confidence interval)Probability distributionSource 
  • *

    Supplied by London Health Protection Agency. ILI, influenza-like illness; GP, general practitioner; RCTs, randomized controlled trials.

Probabilities
 ILI is influenza0.460.425–0.494Normal on Log of odds (μ = −0.16, SD 0.07)[*]UK surveillance
 Influenza is influenza A0.6840.484–0.884Beta (α = 13.5, β = 6.25)[22]UK surveillance
 Consulting the GP0.282 (0.145–0.533)*   
 Annual excess GP
  consultations due to influenza
623,520388,779–858,263Lognormal (μ = −13.3, SD 0.19)[23,25]UK surveillance
  Size of UK population33,743,500  [24]UK census
  Influenza attack rate0.0660.037–0.112Normal on log of odds (μ = −2.66, SD 0.30)[4]Random effects meta-analysis of RCTs
 Consulting GP within 48 hours
  (conditional on consulting)
0.178(0.150–0.212)   
  Consult on day 10.1090.088–0.135Normal on log of odds (μ = −2.1, SD 0.12)[26]UK surveillance
  Consult on day 20.0930.037–0.117Normal on log of odds (μ = −2.28, SD 0.13)[26]UK surveillance
  Rapid onset of symptoms0.5110.305–0.714Normal on log of odds (μ = 0.05, SD 0.44)[4]Random effects meta-analysis of observational studies
 Receive drug if after 48 hours0.028(0.015–0.045)   
 Dependent on same variables
 as previous parameter
     
 Receive drug if before 48 hours0.9520.775–1.000Beta (α = 10, β = 0.5)Assumption 
 Antibiotic at first visit—no
 treatment strategy
0.420.417–0.423Normal on Log of odds (μ = −0.32, SD 0.007)[23]Observational study
 Antibiotic at first visit—antiviral  strategies0.0480.000–0.171Beta (α = 0.5, β = 10)Assumption 
 Complication requiring return  GP visit—no treatment/ amantadine0.3710.362–0.380Normal on Log of odds (μ = −0.53, SD 0.02)[26,27]UK surveillance
 Relative risk of complication— zanamivir0.7410.575–0.954Normal on log of relative risk
(μ = −0.30, SD 0.13)
[4]Random effects meta-analysis of RCTs
 Relative risk of complication— oseltamivir0.4230.160–0.930Normal on log of relative risk
(μ = −0.86, SD 0.38)
[4]Random effects meta-analysis of RCTs
 Pneumonia—no treatment/ amantadine0.0130.008–0.020Normal on log of odds (μ = −4.35 SD 0.23)[4]Random effects meta-analysis of RCTs
 Relative risk of pneumonia— zanamivir0.350.112–1.094Normal on log of relative risk
(μ = −1.05, SD 0.58)
[4]Random effects meta-analysis of RCTs
 Relative risk of pneumonia— oseltamivir0.150.060–0.720Normal on log of relative risk
(μ = −1.90, SD 0.78)
[4]Random effects meta-analysis of RCTs
 Antibiotic if complication  develops0.8140.806–0.822Normal on Log of odds (μ = 1.48, SD 0.03)[25]Observational study
 Odds ratio adverse event— amantadine1.1580.870–1.620Normal on log of odds (μ = 0.17, SD 0.16)[4]Random effects meta-analysis of RCTs

Effectiveness.  Overall effectiveness of each of the four strategies was expressed in terms of Quality Adjusted Life-years (QALYs) which were calculated as a combination of direct influenza related illness, pneumonia and, in the case of amantadine, adverse drug events.

Four RCTs of oseltamivir versus placebo included patients' own reports of health-related quality of life (HRQoL) (WV15670 [15], WV15671 [16], WV15730, M76001) over a period of 21 days. Mean treatment and placebo data were made available to us by the manufacturer for the four trials combined. In the absence of other HRQoL data for influenza, the trial data were used both to estimate the number of quality-adjusted life-days (QALDs) with no treatment and the additional QALDs generated by oseltamivir treatment. For these two arms of the model, direct quality of life evidence was therefore used. For the other two arms of the model (amantadine and zanamivir), no comparable data exist. Therefore, we combined quality of life data from the four oseltamivir trials with data on the length of influenza illness for zanamivir and amantadine to estimate QALYs. This is explained in more detail below.

The four trials consisted in excess of 1500 otherwise healthy adults who were asked to complete a visual analog scale that was marked between zero and 10. Zero on the scale was labeled “worst imaginable health state” and 10 was labeled “normal health for someone your age.” In a previous study using these data, O'Brien et al. [13] simply transformed the scores that patients gave onto a zero to one scale. Given the labeling of the scales, however, this was not considered appropriate. Two transformations were therefore made to the raw data—first, “normal health for someone your age” was considered equivalent to the mean valuation of own health from a large UK survey [17,18]. Second, we transformed these data totime-trade-off equivalent values using an algorithm reported by the same study. For influenza-positive patients who received placebo, the mean QALDs generated over the 21-day period were 15.13. The mean additional QALDs generated by oseltamivir was 0.37.

For amantadine and zanamivir no comparable data was available and length of illness was therefore used to calculate QALDs. The mean length of influenza illness was calculated for each of the four treatment options and the QALDs generated under the zanamivir and amantadine strategies calculated as a direct proportion of length of illness:

image(1)

The length of illness for true influenza cases receiving either no treatment or ineffective treatment (antiviral after 48 hours of symptom onset, amantadine for influenza B) was calculated as 7.7 days (95% confidence interval [CI] 6.86–8.60) by random effects meta-analysis of influenza-positive patients receiving placebo in all neuraminidase inhibitor (NI) trials [4]. These eight trials cover a number of years, and therefore influenza strains, and were thus considered an appropriate source of data of the true long-term mean duration of symptoms. The reduction in length of illness for influenza-positive patients receiving zanamivir was estimated to be 1.68 days (95% CI 0.81–2.58), based on random effects meta-analysis of five RCTs, and the equivalent figure for oseltamivir, based on three RCTs, was 1.92 days (95% CI 0.94–2.92). Trial data for amantadine come from studies over 30 years old and incorporation of these data into the decision model was hampered by the use of duration of fever as an outcome measure rather than influenza illness [19]. Therefore, a meta-regression was constructed from all oseltamivir RCTs (healthy adults, at-risk adults, and children), which allowed the relationship between symptom and fever duration to be estimated [4]. We then used random effects meta-analysis of amantadine trials that used the currently recommended dose (100 mg/day) to estimate the mean reduction in duration of fever. The estimated mean reduction in length of illness for amantadine treatment was 1.28 days (95% CI 0.81–1.73).

Little reliable data on influenza-related pneumonia are available routinely and therefore the baseline rate of pneumonia was based on random effects meta-analysis of the placebo arms of NI RCTs. This rate of 0.013 (95% CI 0.008–0.02) was adjusted by the relative risk of pneumonia from random effects meta-analysis in the relevant trials of oseltamivir (0.15, 95% CI 0.06–0.72) and zanamivir (0.35, 95% CI 0.11–1.09). No evidence of any such effect was found in relation to amantadine and it was considered credible to exclude this from the amantadine strategy. The valuation of avoided pneumonia was drawn from Murray and Lopez [20].

Adverse events associated with neuraminidase inhibitors have been shown to be extremely mild and rare [4]. Conversely, amantadine is associated with a more severe adverse event profile, including events relating to the central nervous system such as malaise, depression, fatigue, vertigo, and “feeling drunk.” These events were accounted for separately in the model. Meta-analysis of two trials that examined amantadine at the now recommended 100-mg dose provided an estimated odds ratio of 1.18 (95% CI 0.87–1.62) [4]. This nonstatistically significant result and associated uncertainty is incorporated in the model. Expert clinical opinion was sought to identify the impact on quality of life of these symptoms by estimating the likely EQ5D score and to estimate a mean duration of symptoms of 5 days.

Data from the Royal College of General Practitioners' (RCGP) network of sentinel practices over several years were used to estimate the probability that ILI is influenza [21]. Clinicians submitted swabs from patients presenting with ILI to the Central Public Health Laboratory, Colindale, for laboratory analysis. Data were provided by the authors relating to periods when the RCGP consultation rates for influenza exceed 50 per 100,000 population, the rate used to define influenza epidemics that occur most seasons. In those aged more than 15 years, the mean rate was 0.46 (95% CI 0.43–0.49) (D. Fleming and M. Zambon, pers. comm.). It should be noted that this figure is substantially lower than that observed in NI RCTs, typically carried out by investigators who apply strict clinical diagnostic criteria.

The probability that influenza is strain A was estimated from 10 years of data published by the Public Health Laboratory Service [Supplied by London Health Protection Agency] (0.68, 95% CI 0.48–0.88).

The mean number of excess GP consultations for ILI during influenza epidemic periods (compared with when influenza is not circulating) over a 10-year period was estimated by Fleming as approximately 600,000 in the healthy adult population [22]. By estimating the size of the UK healthy adult population [23] and the symptomatic influenza attack rate from random effects meta-analysis of placebo recipients participating in nine double-blind, placebo-controlled influenza prevention trials [4], it was possible to estimate the expected number of symptomatic influenza cases per annum and thereby the proportion of those who consult the GP: 600,000 divided by 2.2 million (34 million × 0.066).

The probability of attending within 48 hours of symptom onset was estimated at approximately 20%, 11% on day 1 and 9% on day 2 [24]. Random effects meta-analysis of published studies [4] was used to estimate that 51% of influenza cases experience rapid onset of illness. We assumed that half of those consulting on day 2 could actually be later than 48 hours of symptom onset because of insidious onset of influenza and therefore unable to benefit from any antiviral treatment. The mean estimate of those presenting within 48 hours of symptom onset was therefore reduced by 2.2% (9% × 0.5 × [1–0.51]). Evidence exists that some patients receive antibiotics at this initial GP consultation [25], but it was assumed that this would substantially reduce where antivirals were prescribed. A beta distribution (α = 0.5, β = 10) was fitted to reflect uncertainty around a mean probability of 0.05.

The baseline probability of an individual developing an influenza complication was estimated as the proportion requiring a repeat GP consultation based on RCGP data [26,27] (0.37, 95% CI 0.36–0.38). The relative risk of developing complications requiring an antibiotic based on NI RCTs was 0.74 (95% CI 0.58–0.95) and 0.42 (95% CI 0.16–0.93) for zanamivir and oseltamivir, respectively [4]. This measure was considered the most realistic proxy measure for repeat GP consultations in NHS practice available from RCTs because repeat consultations themselves were often part of the study protocol.

No evidence of reductions in complications of any type was identified for amantadine. Amantadine does have a relatively poor evidence base compared with the newer NIs. Nevertheless, it was also considered the most credible scenario that amantadine does not impact on repeat GP consultations, antibiotic use, or pneumonia. It can be observed that amantadine is little used in practice at least in part because of concerns over its lack of effectiveness.

Costs.  Cost data, also shown in Table 1 and expressed in 2001 prices, were drawn predominantly from UK published sources [28,29]. All drug costs were inflated to include pharmacy prescribing fees and container allowances (Prescriptions Pricing Authority, pers. comm.).

Characterization of Uncertainty

In all meta-analyses, a random effects model was used and it is the uncertainty around these pooled means that is reflected in the probability distributions. This is to reflect the fact that the model is concerned with mean costs and effects over a long period. Some parameters, such as the probability that ILI is influenza, will vary from year to year and using advanced surveillance or near patient tests may increase the ability to predict on an annual basis. In such a situation, predictive distributions may be more appropriate [30], but this was not the aim of the modeling presented here.

Parameters were considered independent. Normal distributions were assigned to log relative risks and log odds ratios. A normal distribution was also considered appropriate for the reduction of length of illness and the difference in QALYs for oseltamivir. A log-normal distribution was fitted to the length of illness and excess GP consultations because this imposes a lower bound of zero with no upper bound and positive skewness. Beta distributions were fitted where a bound of zero to one was considered appropriate, i.e., for the QALY more than 21 days without treatment and for some probabilities.

Analysis

The model was evaluated probabilistically. Monte Carlo simulation was used to generate a large number of random samples from the parameter probability distributions. Cost-effectiveness acceptability curves (CEACs) were generated to reflect the uncertainty inherent in the model parameters. The costs, benefits, and net benefits of each strategy were calculated for each of the Monte Carlo simulations by the following equation:

image

Where λ represents the maximum acceptable incremental cost-effectiveness ratio (ICER) or “willingness to pay” threshold; Q(Ti) is the expected health benefit of treatment strategy Ti, and C(Ti) is the expected cost of treatment strategy Ti[31]. A strategy is considered optimal if it generates the highest mean net benefit. The CEAC plots the proportion of simulations for each strategy that generate the maximum net benefit across a λ range of £0 to £80,000, thereby providing a clear link between the uncertainty in model parameters and the outputs of the model. That is, for any λ, the CEAC displays the probability that a given strategy is cost-effective (incrementally). The CEAC frontier plots the extent of uncertainty associated with the optimal strategy (i.e., the strategy generating the greatest mean net benefit) across a similar range of λ-values [32].

VOI.  Decision-makers must make a choice as to which of the four treatment strategies should be adopted at the current time despite uncertainty in cost-effectiveness estimates arising from uncertainty in input parameters. The true cost-effectiveness of any strategy may be different from the values estimated in the model and therefore decision-makers may make choices that are actually “incorrect.” A second decision that may be taken with respect to any technology is to attempt to reduce the level of uncertainty by commissioning additional research. VOI provides a rational framework within which that research can be prioritized on the basis of the expected value of reductions in particular elements of uncertainty in the decision analytic model.

One branch of VOI is expected value of perfect information (EVPI) analysis. EVPI is a Bayesian approach that works by taking current knowledge (a prior probability distribution), adding in proposed information to be collected (data), and producing a posterior distribution (synthesized probability distribution) based on all available information. The collection of some types of additional data may lead to a different adoption decision, which is the recommendation of a different strategy to that which would be recommended based on current information. For other additional data the adoption decision may not change. EVPI is obtained by simulating additional information, considering whether this data would result in a revised adoption decision, and quantifying the net benefit of our original baseline decision as compared with the extra net benefit provided by the revised decision given new data.

Two types of EVPI analysis are reported here. Global EVPI provides an estimate of the value of eliminating all uncertainty in the model and therefore represents a ceiling value on the cost of future research. Partial EVPI estimates the value of eliminating all uncertainty associated with individual parameters or groups of parameters, providing an upper boundary for the cost of additional research into specific issues. Given the wide range of data sources used in this model, partial EVPI is particularly useful.

The two-stage integration required for EVPI on subsets of parameters [7] was implemented as a two-stage, nested, Monte Carlo algorithm (see, e.g., Tappenden et al. [31]). A “short cut” approach, which does not require an outer loop of simulation, may be appropriate for many decision tree models with independent parameters. Although parameters are independent throughout in this model, the relationship with net benefit is not linear for all parameters and for this reason the two-stage approach was implemented [7,33].

Uncertainty is characterized to reflect two issues: true data uncertainty and also variability. For example, the proportion of ILI that is influenza is dependent on the strain of influenza, which varies from year to year. No amount of data could correctly predict the strain that may emerge in the next year. There is therefore a degree of variability as well as uncertainty. This analysis does not distinguish variability from uncertainty and therefore the EVPI estimates presented relate to both. Nevertheless, a policy decision taken by a decision-maker such as NICE is expected to hold over a period of many years and that decision should therefore be based on the long-run mean value, in the absence of predictors of intensity of epidemic from year to year.

EVPI is dependent on the size of the population and the timescale over which the technology is relevant. The number of persons developing ILI during the influenza epidemic season can be estimated as the size of the UK healthy adult population [23] (34 million) × influenza attack rate [4] (0.066) × cases of ILI per case of influenza (D. Fleming and M. Zambon, pers. comm.) (1/0.46). This provides a central estimate of 4.1 million cases per annum. Results are presented for assumed 5-, 10-, and 15-year time horizons, with values generated in future years discounted at 6% per annum.

Results

Cost-Effectiveness

Table 3 presents the mean costs, benefits and cost-effectiveness ratios, and associated 95% CIs per person with ILI. Differences in benefits (QALDs) between strategies are relatively small because for each strategy only a small proportion of patients actually receive effective antiviral treatment (either they do not visit the GP at all or they do so after 48 hours, they do not have influenza, or they do not receive antiviral treatment). The cost-effectiveness ratios indicate that for each additional QALY generated by the amantadine strategy compared with no treatment, an additional cost of £11,000 is incurred. Both of the neuraminidase inhibitors are more effective and more costly than amantadine. Oseltamivir dominates zanamivir and generates additional QALYs at £32,000 compared with no treatment and £44,000 compared with the next most effective treatment that is not dominated (amantadine).

Table 3.  Base case costs and QALYs
StrategyMean
cost (£)
(95% CI)Mean
QALD
(95% CI)Cost per QALY-
compared with
no treatment (£)
Cost per QALY (£) (ICER)
  1. ICER, incremental cost-effectiveness ratio; QALD, quality-adjusted life-day; QALY, quality-adjusted life-year.

No treatment10.99(7.55–16.66)6.925(6.358–7.494) 
Amantadine11.09(7.60–16.85)6.929(6.359–7.498)10,82410,824
Zanamivir12.27(8.20–18.99)6.935(6.366–7.505)49,405Dominated
Oseltamivir11.88(8.00–18.27)6.936(6.367–7.503)32,40644,420

The CEACs are plotted for the λ range £0 to £80,000 in Figure 2. At each value of λ the four individual CEACs (one for each treatment strategy) indicate the proportion of the 10,000 Monte Carlo simulations in which that strategy was optimal, i.e., generated the maximum net benefit. Where λ is equal to £30,000, for example, it can be seen that the probability that no treatment, amantadine, zanamivir, or oseltamivir are the optimal strategy is approximately 0.20, 0.74, 0.02, and 0.05, respectively. Note that because these are mutually exclusive options, these proportions sum to unity: only one strategy can be optimal in a single simulation. The CEAC also highlights that, where λ is £40,000, the probabilities that amantadine or oseltamivir are the optimal strategies are approximately equal (0.49 and 0.42, respectively).

Figure 2.

Cost-effectiveness acceptability curve—base case analysis.

The CEAC frontier in Figure 2 illustrates that where λ is less than £11,000, the optimal strategy is no treatment. Between £11,000 and £44,000 the optimal strategy is amantadine, and where willingness to pay per QALY exceeds £44,000, oseltamivir is the optimal strategy.

EVPI

Global EVPI was calculated as the difference between expected net benefit given full information (all parameters resolve at their mean values) and the expected net benefit given current information [31]. Figure 3 plots the global EVPI per person as a function of λ between £0 and £80,000. Results are presented for time horizons of 1, 5, 10, and 15 years. There are local peaks around the mean ICERs for amantadine (£11,000) and oseltamivir (£44,000) and the functions rise again as λ rises. Although the second of these peaks indicates the global EVPI for the population lies between £9.6 million (15-year lifetime) and £0.9 million (1-year lifetime), these values are much lower at conventionally accepted values of λ. For example, where λ is equal to £30,000, the global EVPI is approximately £2 million (15-year lifetimes).

Figure 3.

Global expected value of perfect information (EVPI) for population—base case analysis.

EVPI for individual parameters identifies the reason for the shape of the global EVPI functions. For each of the uncertain parameters, EVPI was calculated as the difference in expected net benefit given full information and expected net benefit given current information about the parameter in question. Figure 4 and Table 4 show the results for those parameters where EVPI was greater than £100,000 for any λ between £0 and £60,000, assuming a 10-year time horizon. These parameters generate values with local maxima at the mean ICER values of each of the three drug interventions, as with the global EVPI.

Figure 4.

Expected value of perfect information (EVPI) for parameters—10-year time horizon.

Table 4.  Expected value of perfect information for parameters—10-year time horizon
 Probabilities
Flu is flu A
Relative risksQuality of life
No treatment
Reduction in length of illness
Pneumonia—oseltamivirComplication—oseltamivirAdverse event—amantadineOseltamivirAmantadineZanamivir
Willingness to pay threshold
 000000000
 10,00089,75900168,571152,005408,536367,5570
 20,00055,4920096,742420,716148,10084,5380
 30,00064,0330074,984741,080227,60600
 40,000449,06200763,6281,017,9291,437,403913,9608,490
 45,000711,455371,140394,1411,115,2981,105,3962,489,0002,057,91767,080
 50,00083,03540,15330,654313,0241,194,0832,102,0781,073,156129,245
 60,0000013,43223,2691,373,1372,263,583291,225310,511

The global maximum occurs at a cost effectiveness threshold of approximately £44,000 for most parameters shown in Figure 4 and Table 4, the ICER associated with oseltamivir. This is the case even for those parameters that relate to the effectiveness of amantadine, e.g., the relative risk of an adverse event as a result of amantadine treatment. This is because the cost-effectiveness of oseltamivir is dependent on the effectiveness of the next best alternative, which in most simulations is amantadine. The EVPI for three parameters continues to rise beyond λ of £60,000. These are parameters that influence the relative cost-effectiveness of oseltamivir compared with zanamivir, namely, the QALYs for no treatment and reduction in illness days associated with oseltamivir and zanamivir. It should be noted that there is substantial variation in the EVPI for each parameter dependent on λ. In many cases, the EVPI is low at a threshold value of £30,000 per QALY.

The EVPI for four parameters exceeds £1 million where λ is equal to £45,000 and a lifetime of 10 years is assumed. These are the relative risk of adverse events for amantadine, the QALYs generated by no treatment, and the reductions in length of illness foroseltamivir and amantadine. The latter two values exceed £2 million.

Discussion

This article reports a study that compares existing and newer strategies for the treatment of influenza in otherwise healthy adults. Data are drawn primarily from meta-analysis of RCTs and, by utilizing best available evidence throughout, are synthesized in a decision analytic cost-effectiveness model.

The model is evaluated probabilistically by specifying uncertain parameters as probability distributions and using Monte Carlo simulation to sample from those probability distributions a large number of times. Mean cost-effectiveness ratios indicate that amantadine is the optimal strategy, assuming λ = £30,000, and that oseltamivir is the optimal strategy where λ exceeds £44,000. CEACs highlight the degree of uncertainty associated with these estimates. If decision-makers are willing to pay £40,000 per QALY, then the probability that amantadine is optimal is very similar to the probability that oseltamivir is optimal.

EVPI extends this analysis of uncertainty by identifying those areas where additional information may be particularly useful. Global EVPI estimates the value of eliminating all uncertainty and thereby serves as an upper bound on the cost of future research. Dependent on the value of λ and the time horizon for the technologies, this value may be as high as £10 million.

EVPI for individual parameters highlights that the uncertainty in many parameter estimates does not play a significant part in decision uncertainty and therefore further studies would not be valuable. In particular, where λ is £30,000 it is unlikely that any further research would be an efficient use of scarce resources. The only exception at this threshold value is the parameter for QALYs on no treatment. The EVPI at this level is £741,000, and because additional information on this parameter could be gathered through observational rather than comparative studies, further research is unlikely to be particularly costly. Some cautions, however, are necessary in relation to potential additional research. The finding may be an artifact of the way the data were analyzed. Given that the current estimate is based on trials of approximately 1500 patients, significant reductions in uncertainty are only likely with extremely large samples. A logical approach would be to examine the existing data in a more disaggregated way than was possible here, e.g., at the trial or at the patient level, before recommending more data collection. The use of expected value of sample information analysis, which estimates the expected value of reducing uncertainty in a similar fashion to EVPI, may be of particular importance if additional data collection is to be considered.

It is only at higher threshold values that the EVPI becomes substantial for a number of parameters that could only be examined further through comparative studies. Of the three parameters that generate EVPI in excess of £1 million at λ = £45,000, one relates to oseltamivir (reduction in length of illness) and two relate to amantadine (reduction in length of illness and relative risk of an adverse event). Further clinical trials of oseltamivir or amantadine may then be warranted, particularly because such studies would inform a number of other parameters such as the relative risk of complications and pneumonia, which themselves generate significant, albeit lower, EVPI. This is, however, only if decision-makers are willing to pay a higher amount per unit of health benefit than is considered customary in the UK.

Other parameters have lower EVPIs. Monitoring of the proportion of ILI that is true influenza and the rate of influenza A to overall influenza is currently undertaken routinely by the Health Protection Agency for surveillance purposes and therefore the low EVPI is less relevant for these parameters. EVPI for these parameters in particular indicates both the value of parameter uncertainty and inherent variability from season to season. Near patient tests, currently considered insufficiently sensitive and excessively costly for routine use in the NHS, could be used to improve the ability of the individual GP to distinguish influenza correctly. Improvements in near patient testing might enable better targeting of antivirals to those able to benefit and thereby increase the benefits of treatment. An alternative approach would be to lower the cost of administering treatments, e.g., by allowing nurse or telephone prescribing, or by allowing sales of NIs “over the counter.”

An additional complicating factor relating to many parameters in the model is that reducing uncertainty has a value for other decisions in addition to the treatment of influenza in the otherwise healthy adult population. These drugs have prophylactic uses, seasonal, postexposure and pandemic, and are also available for other patient groups (at-risk adults and, in the case of oseltamivir, children). One of the future challenges for VOI analysis is the development of methods that recognize the interaction between different decisions.

EVPI is dependent on estimates of the size of the population, the time horizon over which technologies are relevant and the discount rate. These parameters are themselves uncertain, although only simple sensitivities relating to the time horizon have been presented here for the sake of clarity.

Furthermore, EVPI is dependent on the specification of the decision model and the characterization of uncertainty, each of which are potentially controversial. Alternative specifications have the potential to impact either solely on the degree of uncertainty in the model and therefore the estimates of EVPI, or on the central estimates of cost-effectiveness. Two examples illustrate.

First, if it is assumed that NIs reduce mortality and hospitalizations in the same proportion as observed reductions in pneumonia cases, the cost per QALY for oseltamivir would reduce to approximately £5000 compared with amantadine and thereby significantly reduce uncertainty (assuming that λ exceeds £20,000 per QALY). Second, QALYs could be estimated by using only the observed baseline QALDs, ignoring data on the additional QALDs on oseltamivir treatment and adjusting according to length of illness for each of the three drug strategies, including oseltamivir. This strategy has no impact on the mean cost per QALY but would change the method by which the length of illness and QALY parameters are used in the model and thereby the impact of uncertainty associated with each parameter. EVPI estimates are altered by such structural changes in the model.

The authors would like to thank Paul Tappenden, Jim Chilcott, and Karl Claxton for their advice on value of information methods. We are extremely grateful for the detailed and helpful comments of five anonymous referees. We are also grateful for support from the NICE Decision Support Unit.

Source of financial support: This work extends a project funded by the UK NICE and the NHS Health Technology Assessment programme.

Ancillary