Economic Evaluations Alongside Effectiveness Trials
Article first published online: 12 OCT 2010
© 2010, International Society for Pharmacoeconomics and Outcomes Research (ISPOR)
Value in Health
Volume 13, Issue 8, pages 865–866, December 2010
How to Cite
Jackson, J. D. (2010), Economic Evaluations Alongside Effectiveness Trials. Value in Health, 13: 865–866. doi: 10.1111/j.1524-4733.2010.00785.x
- Issue published online: 7 DEC 2010
- Article first published online: 12 OCT 2010
Effectiveness is a concept central to Health Technology Assessment (HTA). Effectiveness is the measure of whether and how well an intervention works to mitigate disease and to enhance and extend human life in normal practice. Effectiveness is “the what” or the noun of HTA. In the current issue, Ridyard and Hughes have conducted an evaluation of how well researchers have studied costs within a selection of 100 primary research articles published in the British journal—Health Technology Assessment. Their evaluation concerns “the how” of economic evaluation alongside clinical trials. They evaluated the methods and means of collecting, describing, and analyzing economic information. As a descriptive article concerning the methods of economic evaluations alongside clinical trials, this work will be useful for health outcomes researchers seeking to optimize the collection of economic information. Clearly, good research techniques serve to strengthen economic findings and make them more credible. But a potentially more important question, not addressed by Hughes, is whether a trial should lead to a companion economic evaluation.
The methods and means of effectiveness research are pivotal considerations in HTA. Effectiveness includes an assessment of efficacy, but it also considers whether a clinical intervention works as well in real world settings as it does in controlled trials. Both evaluative settings are needed for robust and credible estimates of benefit and risk in actual practice. Methodological concerns are central to effectiveness estimates. Regulatory authorities, including the Food and Drug Administration and the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, have issued the E10 Guidance (Choice of Control Group in Clinical Trials) . The guidance describes the types and appropriateness of trial designs and provides a wealth of methodological considerations. Furthermore, the CONSORT initiative (http://www.consort-statement.org) has issued guidance to enable readers and researchers to critically appraise and interpret clinical trials, and they have updated their recommendations to include pragmatic trials—or effectiveness considerations . Why are these references important to economic evaluations of clinical trials and HTA? If one thinks of effectiveness as a noun and economic evaluation as an adjective, then they should align to optimize clarity and perhaps appropriateness. A proper estimate of effectiveness is the foundation for sound HTA decisions.
In 2008, a position paper was issued, entitled: National Institute for Health and Clinical Excellence (NICE): How does it work and what are the implications for the United States? The article was authored by Sorenson, Drummond, Kanavos, and McGuire and is available through Michael Drummond . This article has a section on “assessment processes” and the evidence normally preferred and accepted by NICE. In the “hierarchy of preferred study designs,” as a reliable basis for relative therapeutic effects, the lowest level is “expert opinion,” level 3 is “observational studies without control,” level 2 is “controlled observational studies,” and level 1 is “randomized controlled trials” (RCTs). “The RCT is the ideal method for measuring treatment effects.” Even though RCTs minimize the potential for bias and generate the most credible estimates of therapeutic effect, the D'Agostinos (Junior and Senior) argue that observational and nonrandomized studies have a role in codifying effectiveness and real-world experiences. They recommend statistical methods involving matching, stratification, and/or covariance adjustments to minimize bias in effectiveness research. Good research designs and statistical methods enable generation of the most credible estimates of effectiveness, which comprehends all levels of clinical evidence.
The story of drug-eluting stents provides an insightful case of how randomized and nonrandomized evidence evolves in an iterative fashion to address clinically important therapeutic questions of effectiveness . Percutaneous coronary intervention with drug-eluting or bare metal stents to support optimal blood flow within the heart has been the subject of significant research over the past three decades. Stenting technology has proved therapeutically meaningful to reduce disease secondary to a heart attack, but not without surprises. The stent devices work best when adjunctive medical therapy is used. It was demonstrated in observational research that discontinuing antiplatelet medical therapy too soon after the stent placement was associated with late stent thrombosis leading to another myocardial infarction [7,8]. After a host of observational research was conducted to codify the late effect, JAMA selected two observational (registry) investigations by Win et al. and Beohar et al. and commissioned Drs. Harrington and Ohman to explore the contradictory and somewhat confusing findings [9,10]. One of their observations was: “While iterative and exploratory science is not suitable for development of clinical guidelines, mature science always is.” They also explored the potential biases in the Win and Beohar investigations and recommended ways to minimize bias to derive a more mature estimate of therapeutic effect.
While demand for economic evaluations alongside clinical trials may be increasing [2,11], serious attention should be exercised concerning whether a trial warrants a companion economic evaluation. The clinical trial may not have a credible design or the observational study appropriate adjustments, and therefore, the associated economic evaluation will be flawed. Investigations with credible effectiveness estimates, well conceived, conducted, and characterized along the lines described in the E10, CONSORT and D'Agostinos publications, will probably deserve economic evaluations. These characteristics are associated with the clinical investigations, not with the economic evaluations. The economic findings are informative or hypothesis-generating. The clinical investigations provide the foundation for effectiveness estimates of benefit and risk. To my knowledge, no primary investigation, designed, sized, and powered to explore comparative economic effects, has ever been conducted; that is why I think of the economic evaluation as an adjective, rather than a noun.
- 2Department of Health and Human Services, Food and Drug Administration. International Conference on Harmonisation: choice of control group in clinical trials. Fed Regist 1999;64:51767–80.
- 4National Institute for Health and Clinical Excellence (NICE): how does it work and what are the implications for the U.S.? Available from: http://eprints.lse.ac.uk/26148[Accessed April 2008., , , .