• cost-effectiveness;
  • economic evaluations;
  • health economic guidelines;
  • reimbursement


  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgments
  8. References

Objective: To classify, summarize, and compare the health economic guidelines (HE) issued in Europe, North America, and Australia to clarify similarities and differences between them.

Materials and Methods: In a literature review HE guidelines were classified according to whether they were 1) formalized, 2) informal, or 3) guidelines for health economic methods. All the guidelines were summarized in a table format according to 15 important methodological aspects. The aspects were compared both within and between the three groups.

Results: A total of 25 guidelines were identified, seven formalized, eight informal, and 10 guidelines for HE methods. The levels of agreement for methodological aspects within groups were 40% to 100%, 25% to 100% and 30% to 100% for the formalized, informal, and HE guidelines, respectively. The formal guidelines were slightly more homogenous than the other groups. The between-group comparison showed that the guidelines were in agreement for about 75% of methodological aspects. Disagreement between guidelines was found in choice of perspective, resources, and costs that should be included in the analysis, and in methods of evaluating resources used.

Conclusion: A harmonization of methodological requirements and recommendations exists both within and between the guideline groups. This review provides information concerning the core of agreements that have been reached. A number of policy implications for various parties, mainly the pharmaceutical industry, were identified.


  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgments
  8. References

In 1992, Australia became the first country to formalize and issue mandatory guidelines for health economic evaluations of pharmaceutical products as a requirement prior to reimbursement [1]. This document was revised in 1995 [2]. There are similar mandatory guidelines in Canada [3], Finland [4], the Netherlands [5], and Portugal [6], as well as from the managed care organization Regence Blue Shield (USA) [7–9]. The newly established National Institute for Clinical Excellence (NICE) has just issued its guidelines [10]. The function of NICE is to appraise the clinical and economic benefits of new and existing technologies in England and Wales notified by the Department of Health and the Assembly of Wales, and to make recommendations to the National Health System (NHS). The impact of NICE is predicted to be significant because they have strongly promoted implementing and disseminating their recommendations. In this respect, their guidelines can be regarded as mandatory. Other countries that have developed principles and recommendations for use in health economic evaluations are Denmark [11], Ireland [12], New Zealand [13], Norway [14], the United States [15–17], and Switzerland [18], but these guidelines are still voluntary.

The primary role of guidelines is to function as input in the pricing and reimbursement process. Therefore, payers have an interest in scientifically consistent studies. To clarify what is really required in undertaking economic evaluations of medical technologies, research teams around the world have developed methodological principles and guidelines for health economic analyses [19–29].

The objective of this review is to: 1) classify, present, and group guidelines issued in Europe, North America, and Australia according to their purpose; 2) provide a comprehensive and structured group-wise summary of the guidelines; and 3) generalize and compare the guidelines according to the classifications given in 1), to clarify both similarities and differences between them. Possible policy implications of the guidelines will be discussed.

Materials and Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgments
  8. References

This review is based on literature from various sources, such as health economic journals and government-issued documents. Guidelines issued by governments were identified by Internet searches; both directly via the web sites of various authorities and indirectly via the search engine AltaVista, as well as via direct contacts with the regulatory authorities. Guidelines published in health economic journals were identified both by a literature search via websites (e.g., PharmacoEconomics:; newsletters) and via AltaVista (search profile: authors, cost-effectiveness, cost-utility analysis, pharmacoeconomic analysis, etc.).

Classification of Guidelines

Guidelines for health economic evaluations have been classified according to their purpose by Drummond [30]. Drummond outlines three purposes of guidelines: 1) as a requirement prior to reimbursement 2) as a statement of methodological standards, and 3) as a statement of ethical standards. In this report we have used Drummond's first two purposes or classifications. Good and transparent methodology, and justification for assumptions made in evaluations, will function to some extent as protection against the unethical conduct of studies. We have therefore disregarded the third classification.

Drummond's first classification was subdivided into two subgroups: formalized (i.e., mandatory) requirements prior to reimbursement, and informal (i.e., voluntary) requirements prior to reimbursement. The first subgroup consists of national guidelines that have been established to control expenditure for health care technologies and ensure that funds are spent in the best possible way. The second subgroup consists of guidelines that thus far need not be followed as a part of the reimbursement process (although health economic data may help).

Based on these definitions, we have used the following classification: formalized guidelines [2–10] (as a requirement prior to reimbursement), informal guidelines [11–18] (a recommendation prior to reimbursement), and guidelines for health economic methods [19–29] (guidelines intended for use in discussing and improving methodology in health economic evaluations). These three groups of classifications will be used when presenting and comparing the guidelines.

The guidelines are presented in table format, both individually and by group, under the headings Background, Methodological Issues and Assistance/Extra Input. The Background section presents the affiliation of authors, year of last revision, year of implementation, purpose, and the target audience. The next section presents some important methodological aspects that researchers ought to keep in mind when conducting health economic evaluations. The issues in the methodological section originate from the theoretical framework described in classical health economic textbooks [31,32] and are presented in the following order: perspective, resource use/costs, outcome measurement, type of analysis, incremental or average cost-effectiveness, treatment comparator, methods of data capture, modeling, time horizon, discounting, sensitivity analysis, reporting results, and financial implication for society. The last section, Assistance/Extra Input, reports whether there are standardized price lists when estimating costs, and if templates are available when compiling results.

In earlier research related to health economic guidelines, other dimensions or aspects have also been considered, e.g., Luce and Simpson [33] discussed meta-analysis and DeVries and Gagnon [34] discussed the issue of when studies should be performed and by whom. Our approach considers cross-sectional variations in health economic guidelines from the perspective of economic theory. Consequently, we do not cover all the interesting and relevant aspects in the intersection between health economics and other aspects of importance for the illumination of health care technologies in society.


  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgments
  8. References

A Tabulated Summary Of Classified Guidelines

A structured summary of the guidelines is presented in Table 1. The guidelines are presented in the following order: formalized guidelines A–G (Table 1a), informal guidelines H–O (Table 1b), guidelines intended to improve health economic studies P–Y (Table 1c).

Table 1a.  Formalized guidelines (A-G)
Origin of guidelines(A) Australia—Common Wealth Department [2].(B) Canada—Ontario [3].(C) Ministry of Social Affairs and Health—Finland [4].(D) Dutch guidelines for pharmaco-economic research [5].(E) The Portuguese Pharmacy and Medicines Institute (INFARMED) [6].(F) Regence Blue Shields—Guidelines for the submission of Clinical and Economic Data Supporting Formulary Consideration [7–9].(G) National Institute for Clinical Excellence—Revised Guidelines for Manufacturers and Sponsors of Technologies making Submissions to the Institute [10].
  1. QoL, quality of life; QALYs, quality of life years; WTP, willingness to pay; SG, standard gamble; TTO, time trade off; VAS, visual analogue scale: EQ5D, EuroQoL 5 dimensions; CMA, cost-minimization analysis; CEA, cost-effectiveness analysis; CUA, cost-utility analysis; CCA, cost-consequence analysis; CBA, cost-benefit analysis; RCT, randomized clinical trial; CONSORT, consolidated standards of reporting trials; BMJ, British Medical Journal.

Affiliation of authorsThe Pharmaceutical Benefit Advisory Committee (PBAC).Ontario ministry of Health.Ministry of Social Affairs and Health.Sickness Funds Council.Academia.Academia and Industry.MEDTAP International.
Year of last revisionNovember 1995.March 1999.1998.Nov 2000.
Year of implementationJanuary 1993. June 1999.July 2000.November 1998.2000
PurposeProvide sponsors with guidance concerning the presentation of clinical effectiveness data and the most appropriate form of economic evaluation.Offer guidance on how to address economic issues beyond unit price listing when submitting new products.Provide a comprehensive assessment of the cost of pharmaceuticals and other alternative therapies as well as the related benefits.Provide guidelines which together represent state-of-the-art methodology for pharmaco- economic research to provide reliable, reproducible and verifiable insight into the therapeutic value of a drug.Present a number of reference principles. Improve the technical ability of the authors to ensure high quality studies.To describe the process used by Regence to collect and review clinical, economic and other health outcomes data as of the plan's drug formulary adoption process.Provide manufacturers and sponsors of technologies selected for appraisal with a basis for preparing their submissions.
Target audienceManufacturers (companies preparing submissions).Manufacturers.Manufacturers.The Minister of Public Health, Welfare and Sport.Authors of studies.Decision-makers within the health care sector.Manufacturers and sponsors.
Methodological issues       
PerspectiveSocietal perspective and health care sector.Health care sector and/or Societal perspective.Societal perspective.Societal perspective.Societal perspective.System wide perspective including Regence patients (third party payer).From the National health system (NHS) and Personal social services (PPS) decision-maker.
Resource use/costsDirect health care costs. The use of indirect costs may be included but is not encouraged.All relevant costs (direct and indirect) with reference to the Canadian health care system.All direct health care costs. Possibly indirect costs.All direct health costs (inside and outside the health care sector) must be included. Indirect costs outside the health care sector should be stated and justified. Productivity losses should be valued by the friction cost approach.All relevant direct and indirect costs that can be associated to the treatment. The only indirect cost regarded as relevant is productivity loss.All relevant direct costs.All relevant direct costs incorporated in the NHS and PPS perspective. Resources used by patients should be reported separately (e.g., time costs).
Outcome measurement—Type of utilities accepted? Type of health status classification system and preference scores?Effectiveness units in natural (patient relevant) units Effectiveness units that are related to the analytic method. Both global- and disease-specific QoL instruments may be used. The selected QoL instrument should be valid, reliable and responsive to differences in health status between individuals.Effectiveness rather than efficacy. Effectiveness units: non-monetary (QALYs), monetary (WTP) and natural units. Only non-disease-specific utility scales acceptable when calculating QALYs. The rationale behind the chosen measurement method (source of QALY weights or method of eliciting WTP) should be stated.Effectiveness rather than efficacy. Effectiveness units: QALYs (measurement methods—SG, TTO or VAS). Generic and specific questionnaires.Effectiveness rather than efficacy (efficacy after-modeling can be used). Use cardinal scale to measure QALYs (SG or TTO). Classification systems: Both general and disease specific (e.g., EQ5D). Contingent valuation to measure WTP. Effectiveness units: natural units, quality of life related units or monetary units.Effectiveness not efficacy. Output measures: e.g., QALYs (such as EQ5D or McMaster health utilities index), life years gained, episode-free days etc.. Final outcomes are preferred to intermediate measures. Intermediate endpoints need to be justified.Long-term clinical effectiveness; morbidity and mortality. Final clinical end-points. When intermediate endpoints are used a strong association between the final and intermediate end-point should be demonstrated. Utility measures should preferably be based on social preferences.
Type of Analysis—most preferred?CMA, CEA and CUA.CCA, CEA, CUA and CBA. No single best method is mentioned.CMA, CEA, CUA and CBA. The choice of method must be justified.CEA or CUA—CBA is not recommended.CMA, CEA, CUA and CBA.No preferred method, e.g., CMA, CEA, CUA or CBA could be used. Type of analysis should be transparent with other components in the analysis.CEA or CUA. Subgroup analysis should be undertaken if cost-effectiveness is expected to vary between groups within the population; distinguish between high risk patients and the general population.
Incremental or average c-e ratiosIncremental cost-effectiveness ratios.Incremental cost-effectiveness ratios.Incremental cost-effectiveness ratios.Incremental cost-effectiveness analysis. Report totals as well.Incremental cost-effectiveness analysis. Report totals as well.Incremental cost-effectiveness analysis.Incremental cost-effectiveness analysis.
Treatment comparatorThe treatment that is prescribed for the largest number of patients (in the same therapeutic class).Comparison with least expensive currently available strategy and/or most commonly used strategy.Comparison with the product which is to be replaced, the best available product or minimum practice.The treatment in common practice—the standard treatment.Common practice and/or the most effective or least costly treatment.Multiple treatment analysis.Should be the most frequently used and/or the most cost-effective treatment; could be a drug or a surgical procedure.
Method of data captureData collected from literature (meta-analysis) where RCTs are used—comparison with the main comparator is the ideal. The listing of comparative RCTs must be complete.RCTs or meta-analysis (in which only RCTs are used) to locate various results.RCTs or Meta-analysis (in which RCTs are used) Selection criteria in the studies used must be described.Data from clinical studies under realistic conditions.Results obtained from RCTs are preferred. Meta-analysis derived from RCTs is accepted.RCTs and secondary data.The ideal is prospective RCT with a naturalistic design. Data based on efficacy.
Modeling (acceptable? analytic horizon?)Modeling may be needed to take into account patient conditions that were not accounted for in the RCT. Markov chain process, decision tree, Monte Carlo simulation.Modeling acceptable to incorporate future lifetime costs and effects. Time horizon should be clearly stated.Modeling is acceptable if the treatment situation is changed and if data reflecting the normal situation are not available.Modeling is needed if the primary data do not provide sufficient insight into long term effects and costs.Extensions from short to long term require modeling. The model population must be specified and representative of the user of treatment.Each model should incorporate a disease-based treatment framework examining (i) total costs before the drug is introduced and (ii) after the drug has been listed (new equilibrium).To bridge between efficacy and effectiveness. The nature of any modeling used should be closely explained.
Time horizonDepends on the treatment pattern—the natural history of the disease.Long enough to structure all relevant outcomes and costs.Should be sufficiently long to capture all relevant future costs and effects.Long enough to capture all relevant aspects of costs and effects.Must capture all costs and consequences regardless of the moment at which they occur.Long enough to capture all relevant aspects (costs and effects) of the disease.Should cover the period of which the main health effects and the health care resources use are expected to be experienced.
Discounting (future costs and outcomes)5% discount rate.5% discount rate.5% as a standard combined with a 0% discount rate.The base is 4%. Also use 0%, 3%, or 5% in a sensitivity analysis.Use 5% as a base and 3% in a sensitivity analysis.Discounting should be undertaken (both costs and benefits). The choice of rates(s) should be given and justified.Discount costs at 6% discount rate and benefits at 1.5% (base case). In sensitivity analysis (i) costs and benefits 6% and (ii) costs 6% and benefits 0%.
Sensitivity analysisPresent incremental ratios as 95% confidence intervals. Vary all main outcome variables in the preliminary (trial based) analysis and major assumptions used in the model.Sensitivity analysis must be used to assess the robustness of the conclusions.If the results or the underlying data are uncertain.When conditions and assumptions are uncertain a sensitivity analysis must be undertaken. At least a univariate sensitivity analysis.All uncertain parameters should undergo a sensitivity analysis. These are to be presented as limits (upper and lower).Should be undertaken to control the robustness of the results.95% Confidence intervals for cost differences if data originate from RCTs. Univariate and multivariate sensitivity analysis when modeling is applied.
Reporting resultsPresent results both in aggregated and in disaggregated form. Direct and indirect costs presented separately.In aggregated and disaggregated form. Direct and indirect costs are to be reported separately.Report in disaggregated detail. Probability tree for relevant alternatives. Reporting must follow a standard structure.Should follow a standard form containing all relevant data in a disaggregated fashion.In disaggregated form to facilitate comparison between studies.A clear presentation of the clinical trial (CONSORT statement). All costs and outcomes should be reported in disaggregated detail. The results for the principal outcomes of each study included in submission should be reported individually in tabular form.
       The resources used by each treatment approach should be reported in natural units. Resource data sources must be clearly cited. Estimates in 95% confidence intervals.
Financial implications for societyEstimate financial implications for the PBS and for government health budgets. Calculate costs (savings) as payments actually made.Products that result in large expenditures for the Ontario drug program will require much more rigorous economic impact-analysis.Financial analysis is not part of pharmacoeconomic research.Budget impact on the NHS system. For new technologies the budget impact over a 3-5 year period.
Extra inputs/Assistance   
Valuation/Price list availableManual of Resource Items and Their Associated Costs (Opportunity cost). Alternatively: DRG lists.Unit prices for the resources used must be estimated in Canada.Treatment practices and costs should be adjusted so as to correspond to the Finnish treatment practices and cost structure.Manual for cost research. Methods and recommended prices for economic evaluations in health care. Edited by The Council for Health Insurance, Amselveen.Valuation should (ideally) reflect opportunity costs of resources used. Standard costing is an appropriate alternative. Construct vectors of costs and quantities separately.Prices (unit costs) and quantities should be reported separately. Methods for the estimation of both costs and quantities should be given.Standard unit values based on average costs for each resource item. Prices in Unit Costs of health and Social Care.
Template availableWorksheet (a framework of 19 questions) for preparation of pharmacoeconomic analysis.Standard format for reporting pharmacoeconomic research available in appendix.Template is available in the appendix.BMJs checklist for authors and referees.Standard reporting form.
Table 1b.  Informal guidelines (H-O)
  1. QALYs, quality of life years; SG, standard gamble; TTO, time trade off; WTP, willingness to pay; QoL, quality of life; CBA, cost-benefit analysis; CEA, cost-effectiveness analysis; CMA, cost-minimization analysis; CUA, cost-utility analysis; RCT, randomized clinical trial; BCBS, Blue Cross-Blue Shield; DRG, disease related groups.

Origin of guidelines(H) Danish Ministry of Health—A Report of Guidelines for Economic Evaluations of Pharmaceuticals [11].(I) Irish Healthcare Technology Assessment Guidelines—Draft version no 2. [12].(J)A Prescription for Pharmcoeconomic Analysis. New Zealand [13].(K) Norwegian guidelines for pharmacoeconomic analysis in connection with application for reimbursement [14].(L) Guidelines for Blue Cross and Blue Shield of Colorado and Nevada [15].(M) FDA Draft Guidelines—Principles for the Review of Pharmacoeconomic Promotion—USA [16].(N) Swiss Manual for the Standardization of Clinical and Economic Evaluation of Medical Technology (second draft) [18].(O) Academy of Managed Care Pharmacy (AMCP)—AMCP's Format for Formulary Submissions [9,17].
Affiliation of authorsAcademia.National Centre for Pharmaco-economics in Ireland.Pharmaceutical Management Agency Ltd/Health Funding Authority (HFA).The Norwegian Medicines Control Authority.University of Colorado.The Division of Drug Marketing, Advertising and Communications (DDMAC).Consulted experts from the Swiss federal office of social security.Consulted academic experts.
Year of last revision1998.Dec. 1999.1999.Nov. 1999.1998.1995.
Year of implementationJan. 2002.1994.1995.Oct. 2000.
PurposeProvide knowledge on how to design health economic evaluations.Provide the Dept. of Health, the GMS payment board and prescribers with info. on the c-e of a health care technology.Provide HFA with means of deciding how to allocate resources to those activities that are most desirable.To obtain pharmacoeconomic information to increase and improve the decision-making process concerning reimbursement.Enumerate pharmacoeconomic reviewing principles.Supported decision-making regarding (among other things) reimbursement by social health insurance.Rationalize the formulary decision process and to support informed decisions to obtain value from pharmaceutical products.
Target audienceShould be mentioned.Decision-makers.Pharmaceutical companies.Pharmaceutical companies.Industry—submitting companies.E.g., pharmaceutical marketers.Multiple (e.g., decision-makers, third party payers, provider institutions).Manufacturers (submitting companies).
Methodological issues        
PerspectiveSocietal perspective.The adopted perspective should be clearly stated and explained.Societal perspective.Societal perspective.System impact perspective.Multiple (e.g., society, third party payer, patient). The perspective must be stated clearly.Multiple (society, third party payer, patient, health care provider).AMCP's health plan (AMCP's patients).
Resource use/costsAll relevant costs associated with the treatment.There are no agreed-upon cost models.All relevant direct medical and indirect medical costs (e.g., patient co-payment, travel time).All relevant costs that can be related to the utilization of the drug should be included. Report indirect costs separately from the direct costs.All relevant direct costs met by the health system.All important resource utilization variables consistent with the stated perspective.All relevant direct and indirect costs.Direct medical services provided by the health plan (direct medical costs).
Outcomes measurement—Type of utilities accepted? Type of health status classification system and preference scores?Output measures: QALYs, life years gained, episode-free days, etc. SG, Rating scale and TTO are preferred when measuring QALYs. It is inappropriate to base the evaluation only upon WTP. This method can, however, be used as a complement.The primary outcome measure(s) should be clearly stated and explained (published evidence).Effectiveness rather than efficacy. Generic measurement instruments reflecting societal preferences (e.g., EuroQol). Calculations of QALYs should based on actual (real-life) measures in a real world setting and not on artificial clinical trial settings.Effectiveness (ideally) efficacy until effectiveness studies are available. Effectiveness unit that is related to the analytic method.Final outcomes. Outcomes must be in a form which makes it possible to monitor the impact of the new product on a budgetary period basis. Efficacy terms should be translated into effectiveness terms.Physical units that represent resources utilized (e.g., physician visits, courses of medication con-sumed) or clinical outcomes (e.g., treatment failures, changes in QoL). A link between final and intermediate endpoints should be documented if intermediate end-points are used. If monetary units are used, the monetary values must be dis-closed. QoL instru-ments should incor-porate all relevant and important domains and provide represent-ative measures of these domains. Evidence concern-ing validity of QoL instruments must be documented.Outcome measures: primary clinical outcome parameters, secondary clinical outcome parameters and intermediary clinical outcome parameters. Both specific and generic instruments can be used to elicit quality of life. TTO, SG and Rating scales can be used to elicit utilities. When a CBA is put into practice in accord with methodological measurement principles—try to get as close as possible to the instruments used in theoretical literature.Final outcomes. Disease specific as well as generic measures (e.g., QALYs). Efficacy has to be translated into effectiveness.
Type of Analysis—most preferred?CEA and CUA.CMA, CEA, CUA or CBA. Generally CEA is preferred.CUA.CMA, CEA and CUA. The choice of method must be justified.No preferred method. The method must be consistent with meeting the cost- and outcome-impact requirement.CMA, CEA, CUA and CBA. CBA is preferred.No single method preferred.
Incremental or average c-e ratiosIncremental cost-effectiveness analysis.Incremental cost-effectiveness analysis. Report totals as well.Incremental cost analysis is of limited application.Incremental cost-effectiveness analysis.Incremental cost-effectiveness when CEA or CUA is applied.
Treatment comparatorState the rationale for a chosen alternative. Explain the alternative in detail.No treatment (doing nothing) as well as close comparators.The most commonly used treatment, the least expensive and/or another comparator that is considered relevant.The drug(s) which the product is expected to replace.Comparators should be disclosed when based on head-to-head studies. Differences between treat-ments must be statistically proven.Current practice and non-intervention. The comparator must be relevant for clinical practice.Relevant comparators. Include a discussion of comparator product(s).
Method of data captureRCTs and secondary data.RCT, meta analysis, observational data and modeling.RCT in combination with real-life data to capture patients with poor compliance.Data from RCTs in combination with register data (real-life). Meta-analysis may also be used.RCTs - results should be interpreted for the BCBS treatment population. Meta-analysis; include inclusion criteria for studies, processes for extracted data and statistical analysis! All collected data must be relevant for the BCBS treatment environment.Methods of data capture that pro-duce an acceptable level of precision. Data on a drugs effect must comply with the internal and external validity criteria. Studies based on literature review should include all relevant studies and employ systematic and documented search methods. Data must be applicable to the US environment.RCTs and meta-analysis.Data must be applicable to the health plans population. Prospective piggy-backed-, naturalistic comparative, retrospective-, or modeling studies.
Modeling (acceptable? analytic horizon?)Modeling is appropriated when data is collected from multiple sources.Details should be given about the model used (e.g., decision tree or regression model). The assumptions should be justified.Modeling acceptable for capturing long term effects of an intervention.May be used to estimate long term effects and costs.Modeling acceptable—assumptions should be presented and justified.Models should be used only when it is impossible to gather data using well-controlled studies.Modeling is appropriate to bridge the gap between efficacy and effectiveness.Disease-based analytic model that depicts: disease, clinical course, primary treatment, options, treatment process, clinical pathway etc. Transparent models; assumptions and calculations must be designed to allow the health plan to investigate the results afterwards.
Time horizonLong enough to capture all relevant aspects (costs and effects) of the disease.Should be clearly described and appropriate to the disease and treatment.Long enough to capture all the differential effects of an intervention.Long enough to capture all relevant aspects of costs and effects.Prevalence period—maximum 3 years.Long enough to capture all relevant aspects (costs and effects) of the disease.Long enough (dependent on the nature of the disease).
Discounting (future costs and outcomes)Discounting should be undertaken. No specific rates are given.The choice of discount rate(s) should be given.All costs and benefits should be discounted. The base rate is 10% (0, 5 and 15% in a sensitivity analysis).The discount rate should be in the 2, 5%–10% range.Discounting is of less importance.The usual discount rates are 2.5%, 5% and 10%. Other choices must be justified. 
Sensitivity analysisShould be undertaken to control the robustness of the results.Altered variables and the ranges over which they are altered should be justified.All uncertain parameters (e.g., discount rate, net cost baseline health status) should be varied in a sensitivity analysis.At all levels of the analysis where parameters and assumptions are uncertain.Sensitivity analysis focused on scenario development.Sensitivity analysis should be examined for both the effects and the economic parameters. Sensitivity analyses on pivotal estimates and assumptions.
Reporting resultsIn disaggregated form to facilitate comparison between studies.Major outcomes should be presented in aggregated (summary form) and disaggregated form.Both in disaggregated and in aggregated form.Report results in disaggregated form. Decision tree and Markov models may be used to visualize the results.Summary of pharmacoeconomic evidence should be reported in spreadsheet format (disaggregated form).Inclusion criteria for resource use and monetary valuation should be clearly pre-sented. Basis for conclusion should be presented. Limitations of the study design and findings, assump-tions, etc. should be presented for the reader.All key parameters must be examined in a sensitivity analysis. Limits of parameter values in the analysis should be justified.In disaggregated detail (tabular form). Separate resource utilization, total costs, total effectiveness, and incremental effectiveness in the report format. Costs should be presented as total net costs of introduction of the new product.
Financial implications for societyThe report should cover all relevant economic issues in a disaggregated form (e.g., cost per case). A discussion of comparability with other studies should be included.System wide impact for the health plan. Impact assessments should be estimated for the first three budget periods.
Extra input/Assistance 
Valuation/Price list availableMethods for estimation of prices and quantities should be presented.Prices based on DRGs.Opportunity costing (ideally). Calculations of costs should reflect Norwegian condition.Average wholesale prices for drug costs and Medicare fee schedules for medical costs.Average Wholesale Price (AWP).Second best opera-tional procedures to determine the costs of a tech-nology have to be applied since the calculation of opportunity costs requires perfect markets. Accept-able cost data: fees, tariffs and reimbursement figures.Prices and resources must be applicable to the health plan.
Template availableShort structure of analysis is presented.A reporting structure is given on p. B3-B4 in the manual.Formulary submission checklist.
Table 1c.  Guidelines intended to improve HE studies (P-Y)
Origin of guidelines(P) A proposal for methodological guidelines for economic evaluation of pharmaceuticals—Belgium [19].(Q) Canada—CCOHTA [20].(R) Guidelines and recommendations for French pharmacoeconomic studies [21].(S) A proposal for Italian guidelines in pharmacoeconomics [22].(T) Economic analysis of health technologies and programs. A Spanish proposal for methodological standardization [23,24].(U) Hanover Guidelines for Economic Evaluation—Germany [25].(V) UK guidance on good practice in the conduct of economic evaluation of medicines - United Kingdom [26].
Affiliation of authorsBelgian Society for Pharmaco-epidemiology.CCOHTA, Department of Health, industry, academia.Colleges Des Économistes De La Santé.The Mario Negri Institute Centre for Health Economics (CESAV).Consulted academic experts.University of Hanover.Government/Pharmaceutical industry.
Year of last revision2nd edition 1997 (November).1994.
Year of implementationNovember 1995.1994.April 1997.1995.1995.January 1995.
PurposeProvide guidance to conductors and evaluators of pharmacoeconomic studies.Provide assistance to the doer of pharmacoeconomic analyses in providing standardized and reliable info to the target audience. Consistent with a variety of purposes.Provide credibility (for the results), quality (studies) and comparability (between studies).Promote the diffusion of a rational approach to health care problems in Italy.Formulate an initial proposal for methodological standards for economic evaluation.Improve clarity of health economic studies.
Target audienceMultiple (e.g., decision-makers, third party payers, provider institutions).Decision-makers on different levels.Decision-makers on different levels. Researchers.Decision-makers on different levels and in various contexts.Decision-makers on different levels.
Methodological issues       
PerspectiveSocietal perspective (ideally), other perspectives should also be considered.Societal perspective.Depends on the aim of the study.Third party payer or patient perspective.Depends on the aim of the study (e.g., societal, third party payer, etc.). The perspective should be stated.Societal perspective.Societal perspective.
Resource use/costsAll relevant costs should be reported. Indirect and intangible costs should be reported separately.All health care costs, direct and indirect, that can be associated with the treatment should be included.The costs included depend on the aim of the study. All costs that are relevant must be distinguished and presented in detail. Report indirect costs separately.Only direct costs are to be considered.Depends on the aim of the study.All direct and indirect costs. Costs of side effects are to be reported with the probability of their occurrence.All relevant direct and indirect costs.
Outcome measurement—Type of utilities accepted? Type of health status classification system and preference scores?Effectiveness rather than efficacy. Both specific and generic instruments can be used to elicit quality of life. When eliciting QALYs, preference measures are recommended (e.g., TTO, SG or rating scale).Effectiveness rather than efficacy. Effectiveness units: QALYs recom-mended. Contingent valuation preferred for assigning values to outcomes in CBA. HrQoL instruments: Functional living index, Western Ontario-McMaster Osteo-arthritis index, Sickness Impact Profile and Nottingham Health Profile.Effectiveness rather than efficacy. The use of final end-points is preferred to surrogate end-points. Generic and disease specific instruments or a combination of these are acceptable for measuring quality of life. QALYs can be elicited through the use of SG, TTO or VAS. The use of WTP must be justified.Effectiveness rather than efficacy. Only clinically relevant end-points should be considered (e.g., survival, quality of life). Surrogate end-points should be viewed with caution.Effectiveness rather than efficacy. The use of more than one measurement of effectiveness (e.g., life years gained and QALYs) is recommended. Generic measures are preferable when eliciting QoL.Valid and reliable profiling and indexing instruments should be used for measuring quality of life. Both indication and/or non-specific instruments can be used.Outcome measures should be identified and the basis for their selection reported. Generic measures of QoL when CUA is used.
Type of Analysis—most preferred?CMA, CEA, CUA and CBA. The choice of method must be justified.CCA, CMA, CEA, CUA and CBA. CUA and CBA most preferred.CMA, CEA, CUA and CBA. The choice of method must be justified.CMA, CEA or CUA. CEA is most preferred.CEA and/or CUA.CMA, CEA, CUA and CBA. The choice of method depends on the purpose of the study.CMA, CEA, CUA or CBA.
Incremental or average c-e ratiosIncremental cost-effectiveness analysis. Reports totals as well.Costs and outcomes must be reported as increments. Report totals for each alternative as well.Incremental cost-effectiveness analysis.Incremental cost-effectiveness analysis.Incremental cost-effectiveness analysis.Incremental cost-effectiveness analysis.Incremental cost analysis.
Treatment comparatorExisting, minimum or most effective practice. The choice of comparator should be justified.Comparison with both existing practice and minimum practice.Current and/or most recommended practice (ideally). Practice in the same pharmacotherapeutic class.The comparator should be used both internationally and in Italy. Can be the most widely used or the doing nothing alternative.Depends on the aim of the study. Possible options are: most efficient, most used or doing nothing.Most effective form of treatment, most widely distributed or minimum practice.Should be stipulated and justified.
Method of data capturePrimary data (RCTs) and meta-analysis can be used. The quality of the data should be documented.Performance and documentation of data collection should follow well-defined procedures.RCTs, retrospective case-control studies, intervention trials, cohort studies, patient databases, modeling, etc. Overview of RCTs in a meta-analysis is also acceptable.RCTs or meta-analysis (based on RCTs).RCTs or meta-analysis. Meta-analysis must be carried out in such a way that reproducibility is guaranteed.Data from clinical trials is recommended. Meta-analysis is also accepted.RCTs, meta-analysis, observational data and modeling. The choice of method should be justified.
Modeling (acceptable? analytic horizon?)The analytic horizon should be extended to capture all relevant outcomes. To do so modeling might be required. The structure and rationale of the model should be explained.The structure and the nature of the model must be presented.Modeling is needed when the availability of data concerning future events is limited.When data that capture the whole analytic horizon are not available, modeling is suggested. The underlying assumption of the model must be well described.In connection with calculation of long term effects and costs.Modeling is acceptable.
Time horizonLong enough to capture long term effects and costs.Long enough to capture all relevant outcomes.Long enough to capture long term effects and costs.The time horizon should be the entire life span of the patients affected.
Discounting (future costs and outcomes)Use 5% as a base. Rates in the 0-5% range are recommended in a sensitivity analysis.Standard discount rate 5%. In sensitivity analysis use 3% as a base and 0% as a minimum.Discount costs using 2.5-5% rates. The effects should be presented with and without discounting.Use an initial rate of 5% (3%-7% in a sensitivity analysis).The standard rate is 6%.5% as a base - use 3% and 10% in a sensitivity analysis.The recommended rate is 6%.
Sensitivity analysisUncertain parameters should undergo a sensitivity analysis. The ranges and choice of parameters should be justified.To capture all kinds of uncertainty should a rigorous sensitivity analysis be undertaken.Both classical sensitivity analyses where uncertain parameters are varied and statistical distribution tests.Must be applied when the economic model is deterministic. Uncertain parameters should be varied in an increasing and a de-creasing range of 10%.Calculate a central baseline and extreme values for an interval (2 standard deviations).Should be undertaken when parameters are uncertain. The upper and lower limits must be justified.All uncertainty in the analysis should be covered with the use of confidence intervals and appropriate ranges for parameters.
Reporting resultsReports in disaggregated and technical detail following a standardized structure.Standardized reporting structure. Reports both in disaggregated and aggregated form. Probability tree for clinical outcomes should be provided for the relevant alternative.Results should be presented in a successive and detailed way. Costs and effects are to be presented separately.The use of various indicators of effectiveness in the report is recommended (Cost per life year gained, cost per QALY gained, etc.).In disaggregated form.
Financial implications for societyA financial impact analysis should be conducted for major organizations that would be affected by the decision (primary decision-maker).
Extra inputs/Assistance
Valuation/Price list availableCosts should ideally be valued in terms of opportunity costs. Physical units are to be reported separately from the costs of the resources used. Resource utilization data must reflect local conditions.Economic definition of costs must be used - the concept of opportunity cost. CCOHTA Guidance document for the costing process.Costs borne by a third party payer should be considered - gross prices including patient copayment and charges. Price list: Italian National Tariff List.Production costs from the accounts of the producer (ideally). Market price under conditions of perfect competition may also be used.Resource use should (ideally) reflect full opportunity cost. Average cost data is acceptable as a proxy.
Template availableStandardized reporting structure is included in the appendix.Standardized reporting structure - a framework to ensure that studies are reported in an adequate and consistent manner.Ten questions which should be asked when designing health economic evaluations (appendix).
  1. QALYs, quality of life years; TTO, time trade off; WTP, willingness to pay; SG, standard gamble; CBA, cost-benefit analysis; HrQoL, health-related quality of life; VAS, visual analogue scale; QoL, quality of life; CUA, cost-utility analysis; CMA, cost-minimization analysis; CEA, cost-effectiveness analysis; RCT, randomized clinical trial; CCOHTA, Canadian Coordinating Office for Health Technology Assessment.

  2. QALYs, quality-adjusted life years; SG, standard gamble; TTO, time trade off; HUI, Health Utilities Index; WTP, willingness to pay; CUA, cost-utility analysis; CMA, cost-minimization analysis; CEA, cost-effectiveness analysis; CBA, cost-benefit analysis; RCT, randomized clinical trial.

Affiliation of authorsAcademia (sponsored by the US Public Health Service).PhRMAUniversity of Pennsylvania.
Year of last revision1996 (published Feb. 1997).1996.1995.
Year of implementation
PurposeAcademic.Provide PhRMA members with a set of voluntary principles for the conduct of pharmacoeconomic research.Standardization of methods.
Target audienceReference case for analysts and researchers.PhRMA members.Decision-makers on different levels.
Methodological issues   
PerspectiveReference case for analysts and researchers.Depends on the aim of the study (e.g., societal, third party payer, etc.). The perspective should be stated.Multiple (e.g., society, third party payer, patients).
Resource use/costsAll relevant direct and indirect costs that can be associated with the intervention.All relevant direct and indirect costs in accordance with the perspective should be included.All relevant direct and indirect costs should be included.
Outcome measurement - Type of utilities accepted? Type of health status classification system and preference score?An outcome measure that incorporates both quality of life and life expectancy (e.g., life years gained) is recommended. QALY weights can be obtained by SG, TTO and/or rating scale (interval scale). A generic health status classification system is recommended - for example HUI, EuroQoL and Quality of Well-Being Scale.A variety of different outcome measures are accepted: monetary benefit, effectiveness, QoL, utility, efficacy, safety, morbidity, mortality.Outcome measures: natural units (e.g., years of life saved, work loss days), QALYs and WTP.
Type of analysis - most preferred?CUA is preferred.CMA, CEA, CUA, or CBA.CMA, CEA, CUA or CBA.
Incremental or average c-e ratiosIncremental cost-effectiveness analysis.Incremental cost-effectiveness analysis.
Treatment comparatorExisting practice is the most appropriate comparator. The best available alternative or no intervention alternative can also be used if the existing practice is not cost-effective.Alternative treatment - other drugs, surgical or doing nothing.Least expensive currently available therapy, most commonly used or most effective therapy.
Method of data captureCorrected RCT results (deleting protocol driven costs or outcomes) and/or meta-analysis.A variety of data sources can be used, for example RCTs and databases. The quality of the collected data must be documented.RCTs (when efficacy is of interest). When the time horizon is longer, other methods are appropriate: large-sample trial, modeling, multivariate analysis, etc.
Modeling (acceptable? analytic horizon?)When the time frame of the study is too narrow to capture all relevant costs and outcomes, modeling is then a valid form.Modeling (Markov, decision analysis, regression, etc.) is appropriate to bridge the gap from efficacy to effectiveness or for adding long term outcomes.When modeling is used, the underlying logic of the model should be described.
Time horizonLong enough to capture all relevant aspects of the treatment.Should be stated and based on the likely use and effect of the drug.The selected time horizon should be justified and sufficient information should be provided for readers to evaluate the time frame.
Discounting (future costs and outcomes)The base rate is 3%. In a sensitivity analysis the rate can be varied within the 0%-7% range.Discounting should be undertaken but no specific rate is given.The base rate is 5%. This should be varied in a sensitivity analysis.
Sensitivity analysisKey parameters should, at a minimum, be varied in a univariate sensitivity analysis. A multivariate analysis may also be conducted for important parameters.Uncertainty should be demonstrated in a sensitivity analysis (e.g., random effects and uncertain assumptions).The particular variables selected for a sensitivity analysis should be described and justified.
Reporting resultsPresent the results in 2 parts, a journal report (publishable report) and a technical report (a more detailed report). The results should be reported in a disaggregated form.The results should be presented in a disaggregated form - readers should be able to reproduce the key results.Report the results in disaggregated form. Cost items should be differentiated as much as possible.
Financial implications for society
Extra inputs/Assistance   
Valuation/Price list availableA resource that is used should (ideally) be valued at its opportunity cost. In case this is not possible, Average Wholesale Price is recommended.The price and the quantity of a consumed resource should be described separately.
Template available 

A Generalized Comparison of Different Groups Of Guidelines

Table 2 is intended to give a general overview of the groups of guidelines and their recommendations, as presented in the previous section. As can be seen, each group has its own column in the table. These columns show the typical guideline within each cluster at each analytical level. A generalization has different levels of strength. If all the guidelines within a group agree about a recommendation, the level of strength is equal to 1 (7/7 = 1, 8/8 = 1 and 10/10 = 1). When there is major disagreement between guidelines and more than half of the guidelines have recommendations other than the typical example, the level of strength is marked as vague.

Table 2.  An overview of similarities and differences between groups of HE guidelines
 Formalized A–GLevel of StrengthInformal H–OLevel of StrengthHE methods P–YLevel of Strength
  1. CEA, cost-effectiveness analysis; CUA, cost-utility analysis; CBA, cost-benefit analysis; c-e, cost-effectiveness; RCT, randomized clinical trial.

Analytical levelMethodological issues
Perspectivei) Societal4/7i) Societal3/8 vagueDepends on the aim of the study5/10
ii) Societal and/or health care sector6/7ii) Multiple (including societal, third party payer2/8 vague
Resources/costsDirect health care costs5/7All relevant costs4/8All relevant costs5/10
Outcome - measurementGeneric and disease specific measurement instruments6/7Generic and disease specific measurement instruments5/8Generic and disease specific measurement instruments8/10
Type of analysisi) CEA or CUA7/7i) CEA or CUA.5/8i) CEA or CUA9/10
ii) CEA, CUA or CBA4/7  ii) CEA, CUA or CBA8/10
Incremental or average c-eIncremental c-e7/7Incremental c-e4/8Incremental c-e9/10
Treatment comparatorCommon practice5/7Common practice3/8 vagueExisting practice3/10 vague
Methods of data captureRCT (and meta analysis)7/7RCT (and other methods)7/8RCT (and other methods)5/10
Time horizonLong enough7/7Long enough6/8Long enough5/10
Discounting5% discount rate4/72.5-10% discount rate2/8 vague5% discount rate5/10
Sensitivity analysisWhen uncertainty arises7/7When uncertainty arises8/8When uncertainty arises10/10
Reporting resultsIn disaggregated detail6/7In disaggregated detail8/8In disaggregated detail8/10
Financial implications for societyNot necessary4/7Not necessary6/8Not necessary9/10
Extra input/Assistance
Valuation/Price list availableCountry specific cost condition3/7No principles of valuation2/8 vagueOpportunity cost pricing5/10
 Price list available3/7Fees and/or tariffs2.8 vague  
TemplateTemplate4/7No template5/8No template7/10

The following one-to-one comparison between groups has been made: 1) formalized versus informal, 2) formalized versus health economic methods, and 3) informal versus health economic methods. To be as concise as possible, the focus is on the most obvious differences between the groups.

Formalized Guidelines versus Informal Guidelines

When the formalized guidelines are compared with the informal guidelines, it is seen that 12 out of 15 levels are fairly similar. The most obvious differences concern 1) the resources/costs that should be included in the evaluation, 2) the choice of discount rate, and 3) the valuation of costs.

The formalized guidelines are quite clear on which costs to include in the health economic evaluation. Five out of seven stress that all relevant direct health care costs should be included in the evaluation, while four out of seven of the informal guidelines recommend that the analyst include all relevant costs, including indirect costs.

Individual time preferences are reflected in the discount rate. The standard discount rate is 5% and is often used in health economic evaluations. The formalized guidelines have adopted this rate in four out of seven cases. Recommendations in the informal guidelines are vague; two out of eight recommend that the analyst adopt a discount rate between 2.5 and 10%.

The final and perhaps most important difference between these groups concerns how to measure costs. The formalized guidelines are divided into two groups of equal size; three out of seven suggest country-specific costing, and three out of seven can provide researchers with price lists. Whether country-specific costing means that costs should reflect the best use of resources (opportunity cost) is not obvious. The informal guidelines are also divided into two groups of equal size; two out of eight suggest no principles of valuation, or fees and tariffs, respectively. There seems to be major disagreement among the informal guidelines as to how to value resources used; for example, Norway advocates opportunity costing whereas Switzerland advocates fees and tariffs.

In summary, the formalized guidelines are generally more homogenous than the informal guidelines.

Formalized Guidelines versus Guidelines for Health Economic Methods

When examining the differences between the formalized guidelines and the guidelines for health economic methods, it is important that their respective purposes be borne in mind. The formalized guidelines were developed to inform decision makers within the regulatory authority as to whether a pharmaceutical product adds enough extra value to be reimbursed. The guidelines for health economic methods were established to develop a state-of-the-art methodology for health economic evaluation based on economic theory.

The formalized guidelines and the guidelines for health economic methods are fairly similar in 11 of 15 levels. However, four main differences can be distinguished between the two types of guidelines. These are: 1) the perspective of the evaluation 2) the recommendations and requirements concerning evaluation of resource use/costs 3) the choice of treatment comparator, and 4) the valuation of costs.

The formalized guidelines stress that the perspective of the analysis should be societal, whereas the guidelines for health economic methods appear to be less restrictive. The guidelines for health economic methods generally point out that the perspective is dependent on the aim of the study. This means that the researcher has more freedom to choose the analytical aspects he/she wants to emphasize within the scope of the perspective. For example, the researcher has the freedom to examine the impact of two pharmaceutical products on the utilization of inpatient care and is allowed to exclude costs that fall outside this area. This approach is narrower than the societal or the health sector approach. The societal perspective stipulated by the formalized guidelines does not allow the researcher to exclude important direct costs within the health care sector, which include utilization of both inpatient and outpatient care. In this respect the formalized guidelines are more restrictive.

Although the guidelines for health economic methods are less restrictive in the choice of perspective, it is interesting to note that these guidelines actually have more restrictive inclusion criteria concerning resources used and costs. Within the societal perspective, all costs, both direct and indirect, should ideally be included to capture the total impact of a pharmaceutical product. This requirement is met by the guidelines for health economic methods but not by the formalized guidelines. Consideration of direct costs outside the health care sector, such as patients' out-of-pocket expenses and transportation costs, and indirect costs for productivity losses, is not a general requirement in the formalized guidelines. The reason for not also including indirect costs as a requirement could be that the method (human capital approach) for measuring these costs is still controversial.

Agreement among the formalized guidelines to use common practice as the treatment comparator is quite clear; the strength of this generalization is five of seven. It is difficult to find a general rule for choice of comparator in the guidelines for health economic methods. A variety of different comparators are suggested, such as existing practice, minimum practice, and most effective practice, but these definitions are quite vague. The most general recommendation is existing practice, which is stressed in 3 out of 10 cases. If the definitions for existing practice and common practice are considered to overlap, there is a weak relationship between the formalized guidelines and the guidelines for health economic methods in this area.

Valuation of health care resources appears to be a difficult problem. A general rule for decision making in formalized guidelines does not exist—both country-specific costing and use of available price lists are recommended—however, the principle on which valuation of resources rests is unclear. In 5 out of 10 cases the guidelines for health economic methods stress that resources should reflect the opportunity cost of the resources used. This principle is a cornerstone in economics, but it is difficult to apply in practice because of imperfections in health care markets.

In summary, there appears to be more agreement among regulatory authorities than among health economic researchers on how to guide those who perform health economic studies.

Informal Guidelines versus Guidelines for Health Economic Methods

When the informal guidelines are compared with the guidelines for health economic methods, it can be seen that 11 levels out of 15 are fairly similar. However, obvious differences are noted in four areas. These are: 1) the perspective of the study 2) the choice of treatment comparator 3) the choice of discount rate, and 4) the valuation of costs. Differences in the choice of discount rate and in the valuation of costs were discussed when we analyzed formalized versus informal guidelines and formalized versus health economic methods. We shall now focus on 1) and 2).

As was already pointed out, the guidelines for health economic methods have adopted a perspective that is dependent on the aim of the study. The informal guidelines have adopted a societal perspective in three out of eight cases and a multiple perspective in two out of eight cases. A logical explanation for this distribution in the informal guidelines could be the differences in health care systems in the countries that have issued these guidelines. For instance, the US FDA guidelines and the Swiss guidelines are both informal guidelines. Both reflect the respective health care systems in these countries. These health care systems are based on a free-market structure wherein a variety of providers and payers of health care coexist as separate bodies. The private (third-party) payers of health care, i.e., insurance companies, focus on direct health care costs that would otherwise be borne by the patient himself/herself. This means that costs falling outside this range are more or less ignored (e.g., costs for workdays lost). On the other hand, in countries like Norway and Denmark that are more regulated, the government plays a major role in governing the health care system; hence, the societal perspective is of more interest.

As was the case when we compared the formalized guidelines and the guidelines for health economic methods, there appears to be some confusion concerning the definitions of alternative treatments. The informal guidelines are vague in their recommendations of common practice as the treatment comparator (3/8), and the guidelines for health economic methods are vague in their recommendations of existing practice (3/10). Are these poorly defined concepts equivalent, or do they partly overlap? If they are equivalent, agreement between the informal and the health economic guidelines is weak. If they only partly overlap, agreement is very weak.

In summary, the informal guidelines and the guidelines for health economic methods seem to be equally generalized.


  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgments
  8. References

This review has classified, summarized, and compared health economic guidelines issued in Europe, North America, and Australia according to their purpose. Three classes of guidelines were identified and compared: formalized guidelines (i.e., mandatory guidelines prior to reimbursement), informal guidelines (i.e., voluntary guidelines prior to reimbursement), and guidelines intended for use in improving and discussing health economic methods.

Fairly good agreement could be found in about three quarters of the guideline recommendations such as type of analysis (cost-effectiveness analysis and cost-utility analysis), incremental cost-effectiveness ratios, acceptance of modeling, and adequacy of time horizon. This indicates good harmonization among the three sets of guidelines concerning core aspects. One possible explanation for this harmonization could be that the formalized guidelines were generally issued earlier than the informal guidelines and therefore could be functioning as a template.

There are some important differences among the three groups, however. For instance, disagreement could be found in the perspective of the study, resources used and costs, and the valuation of resources used in the analysis. The perspective of the study and the choice of resources and costs used vary because of differences in health care systems or in the purposes of the guidelines. The monetary valuation of resources and costs used seems to vary because most guidelines do not have standardized recommendations for valuation, such as price lists.

Methodological recommendations concerning guidelines have been reviewed earlier [33–38]. However, reviews must be updated regularly as more countries continue to develop guidelines, and the status of existing and newly issued guidelines is ambiguous. In some countries guidelines are a formal requirement, whereas in others they are still used on a voluntary basis.

This review contains a substantial number of health economic guidelines. It is, however, not implausible that we have missed some guidelines, for example, guidelines published in a non-English journal. We have also intentionally excluded literature on health economic evaluation with textbook approaches [31,32] and guidelines specifically intended for internal use by an organization. In addition, some early guidelines have been withdrawn or are simply outdated.

Guidelines are likely to affect both the producers and the consumers of economic evaluations. We will focus here on some aspects that might be important to address in the interpretation and implementation of guidelines for the parties involved.

Most of the formal guidelines state that their view is societal, but they nevertheless insist on focusing primarily on direct health care costs. One consequence of not recommending that indirect costs also should be included is that they increase the potential difficulty of proving the cost-effectiveness of certain classes of newly developed drugs. Advantages in aspects of productivity do not bear the same weight as advantages in the health care sector. Future research might change this, since in general economic terms changes in productivity are constantly in focus, i.e., increases in gross domestic product are usually major news.

The principles for monetary valuation of resources are unclear, or at least not explicit, in most guidelines. For purposes of comparability, these need to be harmonized. Harmonization with respect to prices, through the establishment of independent databases, is one way to apply an external standard. Another way is to communicate with customers (e.g., third-party payers, authorities) a priori about which prices are appropriate to use.

The comparators in cost-effectiveness analyses ought to relate to plausible substitutes under normal practice. However, how to implement this in multinational clinical trials is not evident, since normal clinical practices may vary substantially even within one country. In many cases it may be difficult to identify the treatments that ought to be included in a study to fulfill the HE requirements of many different countries.

Most guidelines demand effectiveness data and not efficacy data. Effectiveness data demonstrate that the evaluated treatment works in normal clinical practice, while efficacy data show that the evaluated treatment has an impact on patients in a structured setting. The increasing demand for effectiveness data has considerable impact on the design of clinical studies, e.g., the number of patients needed to be able to show a significant difference between treatments when the patient groups are somewhat heterogeneous.

Harmonization of outcomes and effectiveness is essential if studies are to be compared. Recommendations concerning appropriate outcome measures might be necessary to allow comparison of different studies with different treatments. One way to increase comparability is through research on patient preferences for various indications. In economics, preferences are treated as rather stable, and research on patient preferences concerning different symptoms might be essential in choosing outcome and effectiveness measures. In this way, conflicting messages could be avoided when products for the same indication are priced or reimbursed.

Economists are not able to discuss with certainty health-related behavior based on normative health economic evaluations. Therefore, decisions based solely on health economic data prior to launch of a treatment cannot constitute the only foundation on which reimbursement decisions are based. Hutton and Maynard [39] have pointed out that “the economic result can never be definitive at that stage.” Economic evaluations alongside closely monitored clinical trials should not be regarded as the state of the art approach for appraisals post launch. After a drug is launched, however, the option of analyzing real-life data becomes available. The real-life impact of a particular drug could be assessed by conducting health econometric analyses, or other kinds of retrospective analyses on administrative databases. This is more in line with mainstream economics than the narrow use of health economics or pharmacoeconomic analyses alongside clinical trials. Alternatively, clinical studies might be replicated with retrospective data from different sorts of databases.

Providing a model has occasionally been the solution when there is a lack of information. Models are accepted and even required in most countries. Given the chronic nature of many diseases, it is reasonable and necessary to conduct research using long-term models that ought to be developed over time, expanded, and disseminated. There might, however, be a trade-off between modeling research and the implementation of prospective health economic analyses and health econometric analyses. In the future, it will be reasonable to expect the provider of formal guidelines to also provide guidance on the types of analyses that are acceptable under different circumstances.

Providing detailed guidelines is cumbersome work, and if they are provided, they run the risk of becoming a cookbook instead of a useful tool for conducting an appraisal. The extent to which guidelines will become a useful tool for the health care sector in different countries in working with issues such as cost-containment and cost-effectiveness, remains to be seen. There are recent data suggesting that the studies received so far by the reimbursement authorities in Australia and the Canadian province of British Columbia may not be optimal [40–42]. It is clear from these data that the pharmaceutical industry needs to work in a more long-term, strategic, and innovative way to support its products with useful, timely, and high-quality health economic data.

Developing guidelines is a dynamic process; the number of formal health economic guidelines is increasing, and the content of the guidelines will most likely be further developed. The development of guidelines will hopefully be the result of the interaction between the interested parties: patient groups, governments, the industry, third-party payers, researchers, etc. These parties usually have conflicting interests. For instance, the reaction of Glaxo Wellcome and the industry toward the NICE decision to not recommend the flu drug Relenza as a treatment attracted worldwide attention. Glaxo Wellcome was said to be considering taking legal actions if NICE refused to recommend Relenza, and the company even threatened to move out of the UK [43,44]. In Canada and Australia also, pharmaceutical companies have brought political and legal pressure on the institutions responsible for judging the relative merits of their products [45,46].

From the industry's point of view it is important to take a constructive part in the development of guidelines and to be given the opportunity to propose ways of improving these guidelines. If not, the credibility of the result may be taxed, since many health economic analyses are presently normative, or the results can be used normatively as a result of the experimental setting of the clinical studies on which they are based.

In conclusion, this review has covered 25 health economic guidelines, which can be considered as fairly homogenous. However, there are differences between guidelines to which pharmaceutical companies should pay attention in an effort to design studies that are as relevant and acceptable as possible. Whether or not the guidelines will be an efficient way of regulating the market remains to be seen. Resources will unavoidably continue to be limited, however, despite the success or failure of the guidelines in their present form. If the guidelines turn out to be unsatisfactory, other health policy instruments will most likely replace them.


  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgments
  8. References

AstraZeneca R & D Lund, Sweden funded this study.


  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and Methods
  5. Results
  6. Discussion
  7. Acknowledgments
  8. References
  • 1
    Australian Commonwealth Department of Health Housing, Community Services. Guidelines for the Pharmaceutical Industry on Preparation of Submissions to the Pharmaceutical Benefits Advisory Committee. Canberra: Government Publication, 1992.
  • 2
    Australian Commonwealth Department of Health Housing, Community Services. Guidelines for the Pharmaceutical Industry on Preparation of Submissions to the Pharmaceutical Benefits Advisory Committee. Canberra: Commonwealth Department, 1995.
  • 3
    Ontario Ministry of Health. Ontario Guidelines for Economic Analysis of Pharmaceutical Products. Toronto: Ontario Ministry of Health, 1994.
  • 4
    Finish Ministry of Social Affairs and Health. Guidelines for Reparation of an Account of Health Economic Aspects. Helsinki: Finish Ministry of Social Affairs and Health, 1999.
  • 5
    Ziekenfondraad. Dutch Guidelines for Pharmacoeconomic Research. Amstelveen: Health Insurance Council (Ziekenfondraad), 1999.
  • 6
    The Portuguese Pharmacy and Medicines Institute. Methodological Guidelines for Economic Evaluation Studies on Drugs. Lisbon: INFARMED, 1998.
  • 7
    Regence BlueShield. Guidelines for the Submission of Clinical and Economic Data Supporting Formulary Consideration (Version 1.2). Seattle, WA: Regence Washington Health—University of Washington, 1997.
  • 8
    Mather DB, Sullivan SD, Augenstein D, et al. Incorporating clinical outcomes into drug formulary decisions: a practical approach. Am J Managed Care 1999;5:27785.
  • 9
    Drummond MF & Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ 1996;313:27583.
  • 10
    National Institute for Clinical Excellence. Revised Guidelines for Manufacturers, Sponsors of Technologies Making Submissions to the Institute. London: National Institute for Clinical Excellence, 2001. Available from:
  • 11
    Alban A, Gyldemark M, Pedersen AV, et al. Economic evaluations of pharmaceuticals (in Danish: Sundhedskonomiske evalueringer af lægemidler). Copenhagen: Schultz, 1995.
  • 12
    National Center for Pharmacoeconomics in Ireland. Irish Healthcare Technology Assessment Guidelines (Draft Version 2). Dublin: National Center for Pharmacoeconomics in Ireland, 1999.
  • 13
    Pharmaceutical Management Agency Limited. A Prescription for Pharmacoeconomic Analysis (Version 1). Wellington, NZ: The Pharmaceutical Management Agency Limited (PHARMAC), 1999.
  • 14
    Norwegian Medicines Control Authority. The Norwegian Guidelines for Pharmacoeconomic Analysis in Connection with Application for Reimbursement. Oslo: Norwegian Medicines Control Authority Department of Pharmacoeconomics, 1999.
  • 15
    Langley PC. Formulary guidelines for Blue Cross and Blue Shield of Colorado and Nevada. PharmacoEconomics 1999;16:21124.
  • 16
    Us Food & Drug Administration. Principles for Review of Pharmacoeconomic Promotion (draft). Washington, DC: Food and Drug Administration (FDA), 1995.
  • 17
    Gricar JA, Langley PC, Luce B, et al. AMCP's Format for Formulary Submissions: A Format for Submissions of Clinical and Economic Evaluation Data in Support of Formulary Consideration by Managed Health Care Systems in the United States. Alexandria, VA. Academy of Managed Care Pharmacy (AMCP), 2000. Available from
  • 18
    Bundesamt für Sozialversicherung. Swiss Manual for the Standardization of Clinical and Economic Evaluation of Medical Technology (second draft). Bern: Bundesamt für Sozialversicherung, 1995.
  • 19
    Belgian Society for Pharmacoepidemiology. A Proposal for Methodological Guidelines for Economic Evaluation of Pharmaceuticals. Brussels: Belgian Society for Pharmacoepidemiology (BESPE), 1995.
  • 20
    Canadian Coordinating Office for Health Technology Assessment. Guidelines for Economic Evaluation of Pharmaceuticals (2nd edn.). Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA), 1997.
  • 21
    Collège des Economistes de la Santé. Guidelines and Recommendations for French Pharmacoeconomic Studies. Paris: Collège des Economistes de la Santé (CES), 1997.
  • 22
    Garattini L, Grilli R, Scopelliti D, Mantovani L. A proposal for Italian guidelines in pharmacoeconomics. PharmacoEconomics 1995;7:16.
  • 23
    General Directorate of Services and Evaluation of Health Technologies and the General Directorate of Insurance and Health Planning of the Ministry of Health and Consumption. Proposal for Standardization of Some Methodological Aspects of Cost-Effectiveness Analyses (CEA), Cost-Utility Analyses (CUA), and Cost-Minimization Analyses (CMA) in the Evaluating of Health Technologies and Programs. Madrid: General Directorate of Services and Evaluation of Health Technologies and the General Directorate of Insurance and Health Planning of the Ministry of Health and Consumption, 1994.
  • 24
    Rovira J & Antonanzas F. Economic analysis of health technologies and programs. A Spanish proposal for methodological standardization. PharmacoEconomics 1995;8:24552.
  • 25
    Graf vd Shulenberg J-M. Hanover Guidelines for Economic Evaluation of Health Services (in German: Hanover Guidelines für die ökonomische Evaluation von Gesundheitsgütern und -dienstleistungen). Hanover: Institut für Versicherungsbetriebs-lehre, Diskussionspapier Nr. 10, January 1995. Die Pharmazeutische Industrie 1995;57:2658.
  • 26
    Government/Pharmaceutical Industry Working Party. UK guidance on good practice in the conduct of economic evaluations of medicines. Br J Med Ec 1994;7:634.
  • 27
    Siegel JE, Torrance GW, Russell LB, et al. Guidelines for pharmacoeconomic studies. Recommendations from the Panel on Cost-effectiveness in Health and Medicine. PharmacoEconomics 1997;11:15968.
  • 28
    Pharmaceutical Research and Manufacturers of America. Methodological and Conduct Principles for Pharmacoeconomic Research. Washington, DC: Pharmaceutical Research and Manufacturers of America (PhRMA), 1995.
  • 29
    Task Force on Principles for Economic Analysis of Health Care Technology. Economic analysis of health care technology. A report on principles. Ann Intern Med 1995;122:6170.
  • 30
    Drummond MF. Guidelines for pharmacoeconomic studies—the ways forward. PharmacoEconomics 1994;6:4937.
  • 31
    Gold RG, Russell LR, Siegel JE, et al. Cost-Effectiveness in Health and Medicine. New York: Oxford University Press, 1996.
  • 32
    Drummond MF, O'Brien B, Stoddart LG, et al. Methods for the Economic Evaluation of Health Care Programmes (2nd edn.). Oxford: Oxford University Press, 1997.
  • 33
    Luce BR & Simpson K. Methods of cost-effectiveness analysis: areas of consensus and debate. Clin Ther 1995;17:10925.
  • 34
    DeVries A & Gagnon JP. Cost-effectiveness evaluation in health care: initiative for a standardized methodology. Managed Care Med 1995;2:2533.
  • 35
    Jacobs P, Bachynsky J, Baladi J-F. A comparative review of pharmacoeconomic guidelines. PharmacoEconomics 1995;8:1829.
  • 36
    Genduso AG & Kotsanos JK. Review of health economic guidelines in the form of regulations, principles, policies and positions. Drug Information Journal 1996;30:100316.
  • 37
    Mullins DC & Ogilvie S. Emerging standardization in pharmacoeconomics. Clin Ther 1998;20:1194202.
  • 38
    Drummond M, Dubois D, Garattini L, et al. Current trends in the use of pharmacoeconomics and outcomes research in Europe. Value Health 1999; 2:32332.
  • 39
    Hutton J & Maynard A. A NICE Challenge for health economics. Health Econ 2000;9:8993.
  • 40
    Hill SR, Mitchell AS, Henry DA. Problems with the interpretation of pharmacoeconomic analyses—a review of submissions to the Australian Pharmaceutical Benefit Scheme. JAMA 2000;283: 211621.
  • 41
    Anis AH & Gagnon Y. Using economic evaluation to make formulary coverage decisions—so much for guidelines. PharmacoEconomics 2000; 18:5562.
  • 42
    Rennie D & Luft HS. Pharmacoeconomic analyses—making them transparent, making them credible. JAMA 2000;283:215860.
  • 43
    Anonymous. NICE rejects Glaxo Wellcome appeal against Relenza NHS ban. Pharmaceut J 1999; 263:56.
  • 44
    Anonymous. A nasty start for NICE [editorial]. Lancet 1999;354 (9187):1313.DOI: 10.1016/s0140-6736(99)00181-6
  • 45
    Schuchman M. Drug firm threatens suit over MD's product review. Toronto Globe and Mail, November 17, 1999.
  • 46
    Wilkingson M. Cut-price medicine put at risk. Sydney Morning Herald, December 1,1999.