SEARCH

SEARCH BY CITATION

Keywords:

  • costs and cost analysis;
  • disease outbreaks;
  • economic evaluation models;
  • human influenza;
  • model parameters;
  • review

Abstract

  1. Top of page
  2. Abstract
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Recommendations
  8. Acknowledgements
  9. Funding
  10. Conflict of interest
  11. References
  12. Supporting Information

Background

In our previous systematic review of economic evaluations of pandemic influenza interventions, five model parameters, namely probability of pandemic, duration of pandemic, severity, attack rate, and intervention efficacy, were not only consistently used in all studies but also considered important by authors.

Objectives

Because these parameters originated from sources of varying quality ranging from experimental studies to expert opinion, this study aims to analyze the variation in values used according to sources of information across studies.

Methods

An analysis of estimated values of key parameters for economic modeling was performed against their different data sources, following the standard hierarchy of evidence.

Results

A lack of good-quality evidence to estimate pandemic duration, pandemic probability, and mortality reduction from antiviral treatment results in a large variation of values used in economic evaluations. Although there are variations in quality of evidence used for attack rate, basic reproduction number, and reduction in hospitalizations from antiviral treatment, the estimated values do not vary significantly. The use of higher-quality evidence results in better precision of estimated values compared to lower-quality sources.

Conclusion

Hierarchies of evidence are a necessary tool to identify appropriate model parameters to populate economic evaluations and should be included in methodological guidelines. Knowledge gaps in some key parameters should be addressed, because if good-quality evidence is available, future economic evaluations will be more reliable. Some gaps may not be fulfilled by research but consensus among experts to ensure consistency in the use of these assumptions.


Background

  1. Top of page
  2. Abstract
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Recommendations
  8. Acknowledgements
  9. Funding
  10. Conflict of interest
  11. References
  12. Supporting Information

The H5N1 and pH1N1 outbreaks that occurred in recent years directed international attention toward the cost-effectiveness of interventions aiming to prevent and control pandemic influenza. As a result, and part of its Public Health Research Agenda for Influenza, the World Health Organization (WHO) commissioned the Health Intervention and Technology Assessment Program (HITAP) of Thailand to conduct a systematic review of preparedness strategies and interventions against pandemic influenza, published in early 2012.[1] By offering a comprehensive framework for efficient allocation of resources – a tool that was lacking until then – the study intended to serve as a reference for the future revision of the WHO pandemic preparedness and response guidance.[2]

In our study, we searched relevant databases as well as screened references and contacted authors up to September 2011. Eligible papers were full and partial economic evaluations including both costs and outcomes, while editorials, reviews, and papers on economic impact or complications were excluded. We selected a total of 44 evaluations for the review. Although in general the methods applied were appropriate, we detected important shortcomings in the quality of evidence used. There were also considerable variations in drug regimens and vaccination protocols. In summary, pharmaceutical interventions ranged from cost-saving to high cost-effectiveness ratios. Combinations of pharmaceutical and non-pharmaceutical interventions were cost-effective compared with vaccines and/or antivirals alone. Reduction in contacts, prevention with antivirals together with school closure demonstrated to be especially cost-effective for all countries. In contrast, quarantine for household contacts was cost-ineffective in all settings. Finally, we provided recommendations on practical issues necessary to improve the quality and generalizability of economic evaluation studies in the future. In particular, we underlined the importance of five model parameters (i.e., probability of pandemic, duration of pandemic, severity, attack rate, and intervention efficacy/effectiveness) that were not only consistently employed in all evaluations but also considered important by study authors.

In this study, our purpose is to describe and analyze the variation in key parameter values employed according to sources of information across studies. These key parameters originate from different sources of varying quality ranging from experimental studies to expert opinion. An investigation into this variation is warranted, with the aim to promote the reaching of consensus on certain important parameters used for future economic evaluations and identify future priority research areas.

Methods

  1. Top of page
  2. Abstract
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Recommendations
  8. Acknowledgements
  9. Funding
  10. Conflict of interest
  11. References
  12. Supporting Information

We conducted a descriptive analysis of five key parameters for economic modeling of pandemic influenza interventions. The identified parameters were cross-tabulated against the different data sources, following the hierarchy developed by Cooper et al.[3] In this hierarchy, different data sources are assessed according to their level of quality: a) clinical effect sizes, b) adverse events and complications, c) baseline clinical data, d) resource use, e) costs, and f) utilities (only in cost utility analyses). Parameter sources are given a rank from 1–6 and 9 in descending order, with rank 1 applied to parameters derived from the best quality sources. In summary, in the case of clinical effect sizes/adverse events and complications, rank 1 is given to meta-analyses of randomized controlled trials (RCTs) or RCTs that directly evaluate comparator interventions and quantify final outcomes; rank 2 is given to similar designs but measuring surrogate outcomes or using placebo as a comparator while measuring final outcomes for each intervention; rank 3 is applied to meta-analyses or RCTs that use placebo as comparator and measure surrogate outcomes; rank 4 is given to observational studies; rank 5 to non-analytic studies; and, finally, rank 6 and 9 are given to expert opinion and cases where the source is not clear, respectively. In the case of baseline clinical data, rank 1 is given to purposely conducted case series/analyses of dependable databases including patients from the study setting; rank 2 is given to similar studies that were conducted recently; rank 3 is given to similar studies conducted recently in a different setting; rank 4 is given when these studies are old or the estimates are derived from RCTs; rank 5 is given to estimates retrieved from other economic evaluations; and ranks 6 and 9 are given in the same fashion as for clinical effect sizes/adverse events and complications.

Results

  1. Top of page
  2. Abstract
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Recommendations
  8. Acknowledgements
  9. Funding
  10. Conflict of interest
  11. References
  12. Supporting Information

Key parameters related to baseline clinical data

Figure 1 illustrates the variation of means of the parameters for attack rate, basic reproduction number (R0), pandemic probability, and pandemic duration. The graph clearly demonstrates a lack of high-quality evidence for pandemic probability and pandemic duration, because authors of all reviewed papers derive these parameters from previous economic evaluations, expert opinion, or unclear sources.

image

Figure 1. Hierarchy of quality of evidence for pandemic duration, probability of pandemic, basic reproduction number, and attack rate parameters. Vertical lines represent the range of values. Square boxes represent average values. The numbers above each line are the number of studies. The hierarchy of evidence is based on Cooper et al.[3]

Download figure to PowerPoint

For pandemic duration, there is only one study using data from a previous economic evaluation of antiviral stockpiling in Singapore, which estimates pandemic duration at 12 weeks.[4] In six studies that estimate pandemic duration from expert opinion, the value varies from 15 to 43 weeks. Regarding pandemic probability, although most studies apply the common belief that a pandemic is expected to occur every 30 years, there is one study where the estimate is unexpectedly five years.[5]

In the cases of attack rate and R0, authors of the reviewed studies select evidence from a wide array of data sources ranging from recent cases series/analyses of reliable administrative databases to expert opinion. Although higher-quality sources tend to provide less variation in estimates, there are not considerable differences in absolute values used.

Key parameters related to clinical effect sizes

As for pharmaceutical interventions, there is a knowledge gap in the value of antiviral efficacy measured as mortality reduction, with all six studies using parameters derived from expert opinion or unclear sources (Figure 2). This is not the case for the estimates of antiviral efficacy measured as reduction in hospitalizations, where authors employ a broader range of data sources. In the studies, the absolute values are also similar. For vaccine efficacy, the variations in estimates observed are large, especially evident in seven studies where authors use expert opinion or estimated from other sources, for example one study based on previous pandemics[6], one study based on seasonal influenza[7], and five additional studies where the source of the estimates is unclear.[5, 8-11]

image

Figure 2. Hierarchy of quality of evidence for intervention efficacy parameters. Vertical lines represent range of values. Square boxes represent average values. The numbers above each line are the number of studies. The hierarchy of evidence is based on Cooper et al.[3]

Download figure to PowerPoint

To assess the effect of the 2009 pandemic on the quality of evidence used in economic evaluation studies, Figure 3 compares the ranking of each parameter in studies conducted before and after the 2009 pandemic. Even though the number of studies is small, the 2009 pandemic seems to provide a positive benefit to the quality of evidence used for two of the parameters, namely attack rate and R0, but not for the other parameters.

image

Figure 3. Comparison of number of studies according to quality of evidence between ex-ante and ex-post studies. Ex-ante = before the 2009 H1N1 pandemic; Ex-post = after the 2009 H1N1 pandemic. The hierarchy of evidence is based on Cooper et al.[3]

Download figure to PowerPoint

Discussion

  1. Top of page
  2. Abstract
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Recommendations
  8. Acknowledgements
  9. Funding
  10. Conflict of interest
  11. References
  12. Supporting Information

From our analysis, we identified an important knowledge gap in three key parameters necessary for economic evaluation studies of pandemic influenza preparedness strategies and interventions. This includes pandemic duration, pandemic probability, and mortality reduction from antiviral treatment. Because there is no high-quality evidence for these parameters, resulting both in a large variation of estimated values used and a high impact on economic evaluation results, this will ultimately hinder cross-study comparisons of economic information to guide policy decisions.

In the cases of attack rate, R0, and reduction of hospitalizations from antiviral treatment, there are not large variations in values used. This may be because there is already high-quality evidence available and that most experts are aware of the existing evidence. For example, because there were two studies conducted by Khazeni et al.[12] and Tuite et al.[13] that analyzed information from U.S. and Canadian administrative databases[14] to estimate R0, and economic evaluations conducted after that year employed an estimate that did not differ significantly from the results of those two studies. Even in three other studies published more recently in 2009[15], 2010[16], and 2011[17], which derived the value of R0 from the authors’ own assumptions, we found these estimates comparable to the results from the two studies mentioned above.[13, 14]

There are some limitations in this study. As this study was derived from a systematic review completed in September 2011, studies published later are not included. Nonetheless, an update of our search strategy in MEDLINE/PubMed covering from September 2011 to September 2012 indicates that, although the scope of settings and interventions has broadened, there are few eligible additional economic evaluations[18-21] and most of the studies are cost analyses.[22-25] We believe that our results are still valid even though these new studies are not included. Moreover, we only reviewed economic evaluation studies and, therefore, can only capture the data sources selected by these studies. If there is better-quality evidence available but not used in these reviewed studies, it is not included in our analysis. Lastly, we employed a hierarchy of evidence developed for health economic evaluations of general diseases. However, pandemic influenza is not a disease where the past is a reliable guidance for the future. Hence, the hierarchy may not always be relevant for all parameters. For example, results from genetic studies that are not population-based may be more reliable in giving an indication of how a pandemic flu strain may drift and attenuate over time than observational studies of the past pandemic.

Recommendations

  1. Top of page
  2. Abstract
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Recommendations
  8. Acknowledgements
  9. Funding
  10. Conflict of interest
  11. References
  12. Supporting Information

Firstly, we found that hierarchies of evidence are a necessary tool to help identify appropriate model parameter estimates to populate economic evaluations. Research funders and health economic evaluation methodological guideline developers should request that researchers select the highest quality data sources as possible according to standard hierarchies of evidence.

Secondly, we also identified a knowledge gap in some key parameters that should be addressed by funders of responsible agencies, who should include them in future research programs. This is because future economic evaluations will tend to have less variation in values of parameters used (e.g., the case of R0), if good-quality evidence is available and utilized.

Finally, it may not be possible to fill some evidence gaps (i.e., pandemic probability and duration) by research and these will need to be addressed by reaching consensus among experts to ensure a better consistency in the use of these assumptions, so that future economic evaluations can be comparable and meaningful for guiding resource allocation decisions.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Recommendations
  8. Acknowledgements
  9. Funding
  10. Conflict of interest
  11. References
  12. Supporting Information

The authors would like to thank the researchers who provided information on grey literature, especially those who shared their papers. In addition, we are grateful to the WHO Global Influenza Program for funding the previous systematic review on which this paper is based and for inviting us to publish in this supplement.

Funding

  1. Top of page
  2. Abstract
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Recommendations
  8. Acknowledgements
  9. Funding
  10. Conflict of interest
  11. References
  12. Supporting Information

The Health Intervention and Technology Assessment Program is funded by the Thailand Research Fund under the Senior Research Scholar on Health Technology Assessment (RTA5580010), the Thai Health Promotion Foundation, the Health System Research Institute and the Bureau of Health Policy and Strategy, Ministry of Public Health. This study did not receive any specific funding.

References

  1. Top of page
  2. Abstract
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Recommendations
  8. Acknowledgements
  9. Funding
  10. Conflict of interest
  11. References
  12. Supporting Information
  • 1
    Perez Velasco R, Praditsitthikorn N, Wichmann K et al. Systematic review of economic evaluations of preparedness strategies and interventions against influenza pandemics. PLoS One 2012; 7:e30333.
  • 2
    World Health Organization. Pandemic Influenza Preparedness and Response: a WHO Guidance Document. Geneva, Switzerland: World Health Organization, 2009.
  • 3
    Cooper N, Coyle D, Abrams K, Mugford M, Sutton A. Use of evidence in decision models: an appraisal of health technology assessments in the UK since 1997. J Health Serv Res Policy 2005; 10:245250.
  • 4
    Lee VJ, Phua KH, Chenm MI et al. Economics of neuraminidase inhibitor stock piling for pandemic influenza, Singapore. Emerg Infect Dis 2006; 12:95102.
  • 5
    Newall AT, Wood JG, Oudin N, MacIntyre CR. Cost-effectiveness of pharmaceutical-based pandemic influenza mitigation strategies. Emerg Infect Dis 2010; 16:224230.
  • 6
    Durbin A, Corallo AN, Wibisono TG et al. A cost effectiveness analysis of the H1N1 vaccine strategy for Ontario, Canada. J Infect Dis Immun 2011; 3:4049.
  • 7
    Prosser LA, Lavelle TA, Fiore AE et al. Cost-effectiveness of 2009 pandemic influenza A (H1N1) vaccination in the United States. PLoS One 2011; 6:e22308.
  • 8
    Sander B, Nizam A, Garrison LP et al. Economic evaluation of influenza pandemic mitigation strategies in the United States using a stochastic microsimulation transmission model. Value Health 2009; 12:226233.
  • 9
    Lugnér AK, Boven Mv, Vries Rd, Postma MJ, Wallinga J. Cost-effectiveness of vaccination against pandemic influenza in European countries: mathematical modelling analysis. BMJ 2012; 12:e4445.
  • 10
    Yarmand H. Cost-effectiveness analysis of vaccination and self-isolation in case of an H1N1 outbreak [Dissertation]. Raleigh, NC: North Carolina State University, 2010.
  • 11
    Piercy J, Miles A. The Economics of Pandemic Influenza in Switzerland. Bern, Switzerland: Federal Office of Public Health, 2003.
  • 12
    Khazeni N, Hutton DW, Garber AM, Owens DK. Effectiveness and cost-effectiveness of expanded antiviral prophylaxis and adjuvanted vaccination strategies for an influenza A (H5N1) pandemic. Ann Intern Med 2009; 151:840853.
  • 13
    Tuite AR, Greer AL, Whelan M et al. Estimated epidemiologic parameters and morbidity associated with pandemic H1N1 influenza. Can Med Assoc J 2010; 182:131136.
  • 14
    CDC Community Mitigation Strategy Team. Interim pre-pandemic planning guidance: community strategy for pandemic influenza mitigation in the United States—early, targeted, layered use of nonpharmaceutical interventions. Atlanta, GA: Centers for Disease Control and Prevention, US Department of Health and Human Services, 2007.
  • 15
    Sander B, Bauch C, Fisman DN et al. Is a mass immunization program for pandemic (H1N1) 2009 good value for money? Early evidence from the Canadian experience. PLoS Curr 2009; 1:RRN1137.
  • 16
    Perlroth DJ, Glass RJ, Davey VJ et al. Health outcomes and costs of community mitigation strategies for an influenza pandemic in the United States. Clin Infect Dis 2010; 50:165174.
  • 17
    Halder N, Kelso JK, Milne GJ. Cost-effective strategies for mitigating a future influenza pandemic with H1N1 2009 characteristics. PLoS One 2011; 6:e22087.
  • 18
    Pershad J, Waters TM. Use of tent for screening during H1N1 pandemic: impact on quality and cost of care. Pediatr Emerg Care 2012; 28:229235.
  • 19
    You JH, Chan ES, Leung MY, Ip M, Lee NL. A cost-effectiveness analysis of “test” versus “treat” patients hospitalized with suspected influenza in Hong Kong. PLoS One 2012; 7:e33123.
  • 20
    Araz OM, Damien P, Paltiel DA et al. Simulating school closure policies for cost effective pandemic decision making. BMC Public Health 2012; 12:449.
  • 21
    Chocontá-Piraquive LA, Alvis Guzman N, De la Hoz Restrepo F. Cost-effectiveness of vaccinating pregnant women against pandemic influenza in Colombia. Rev Panam Salud Publica 2012; 31:447453.
  • 22
    Gottschalk R, Bellinger O, Stadthagen S et al. Costs of the influenza pandemic for the public health services - calculations based on the model of the metropolitan region Frankfurt am Main. Pneumologie 2011; 65:697704.
  • 23
    Giglio ND, Castellano VE, Ruttimann RW, Vidal GI, Gentile A. Impact on the direct medical cost related to the influenza virus during 2009 in children under 5 years compared to the period 2006-2008 in a pediatric hospital. Arch Argent Pediatr 2012; 110:1926.
  • 24
    Galante M, Garin O, Sicuri E et al. Health services utilization, work absenteeism and costs of pandemic influenza A (H1N1) 2009 in Spain: a multicenter-longitudinal study. PLoS One 2012; 7:e31696.
  • 25
    Zarogoulidis P, Glaros D, Kontakiotis T et al. Health costs from hospitalization with H1N1 infection during the 2009–2010 influenza pandemic compared with non-H1N1 respiratory infections. Int J Gen Med 2012; 5:175182.

Supporting Information

  1. Top of page
  2. Abstract
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Recommendations
  8. Acknowledgements
  9. Funding
  10. Conflict of interest
  11. References
  12. Supporting Information
FilenameFormatSizeDescription
irv12091-sup-0001-TableS1-S4.docWord document696K

Table S1. Hierarchies of data sources according to quality of evidence (adapted from Cooper et al.).

Table S2. Model parameters of infectivity and disease severity values with quality of evidence (QoE) ranking of reviewed studies.

Table S3 Model parameters of intervention effectiveness values with quality of evidence (QoE) ranking of reviewed studies.

Table S4 Model parameters of pandemic duration and probability values with quality of evidence (QoE) ranking of reviewed studies.

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.